2019-12-08T12:16:20.975 INFO:root:teuthology version: 1.0.0-8b630f9 2019-12-08T12:16:20.976 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.5N_615.tmp -- --verbose --lock --description rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi --block --owner scheduled_ofriedma@teuthology --archive /home/teuthworker/archive/ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4581813 2019-12-08T12:16:21.021 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-08T12:16:21.081 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4581813 branch: wip-ofriedma-dynamic-disable description: rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: ofriedma@redhat.com first_in_suite: false job_id: '4581813' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-ofriedma-dynamic-disable ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 1351c819e1a9518cde4c8b12d645782009028360 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: notcmalloc sha1: 1351c819e1a9518cde4c8b12d645782009028360 openssl_keys: rgw.client.0: ca: root client: client.0 embed-key: true rgw.client.1: ca: root client: client.1 embed-key: true root: client: client.0 cn: teuthology install: - client.0 - client.1 key-type: rsa:4096 rgw: client.0: ssl certificate: rgw.client.0 client.1: ssl certificate: rgw.client.1 compression type: random ec-data-pool: true frontend: beast storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-ofriedma-dynamic-disable sha1: 1351c819e1a9518cde4c8b12d645782009028360 owner: scheduled_ofriedma@teuthology priority: 30 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 1351c819e1a9518cde4c8b12d645782009028360 suite: rgw suite_branch: wip-ofriedma-dynamic-disable suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 1351c819e1a9518cde4c8b12d645782009028360 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.22739 2019-12-08T12:16:21.081 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa; will attempt to use it 2019-12-08T12:16:21.082 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks 2019-12-08T12:16:21.082 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-12-08T12:16:21.083 INFO:teuthology.task.internal:Checking packages... 2019-12-08T12:16:21.097 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '1351c819e1a9518cde4c8b12d645782009028360' 2019-12-08T12:16:21.097 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-08T12:16:21.097 INFO:teuthology.packaging:ref: None 2019-12-08T12:16:21.097 INFO:teuthology.packaging:tag: None 2019-12-08T12:16:21.097 INFO:teuthology.packaging:branch: wip-ofriedma-dynamic-disable 2019-12-08T12:16:21.098 INFO:teuthology.packaging:sha1: 1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:16:21.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-ofriedma-dynamic-disable 2019-12-08T12:16:21.267 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7900.g1351c81 2019-12-08T12:16:21.268 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-12-08T12:16:21.279 INFO:teuthology.task.internal:no buildpackages task found 2019-12-08T12:16:21.279 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-12-08T12:16:21.294 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-12-08T12:16:21.295 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-08T12:16:21.342 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:16:31.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:16:41.371 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:16:51.389 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:17:01.407 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:17:11.424 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:17:21.443 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:17:31.466 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:17:41.480 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:17:51.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:18:01.511 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:18:11.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:18:21.545 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:18:31.561 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:18:41.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:18:51.596 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:19:01.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:19:11.630 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:19:21.647 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:19:31.665 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:19:41.681 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-08T12:19:51.696 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:20:01.712 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:20:11.729 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:20:21.748 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:20:31.763 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:20:41.782 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-08T12:20:51.798 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:21:01.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:21:11.830 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:21:21.845 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:21:31.862 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:21:41.877 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:21:51.893 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:22:01.908 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 0)... 2019-12-08T12:22:11.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 0)... 2019-12-08T12:22:21.944 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:22:31.961 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:22:41.976 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:22:51.993 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:23:02.009 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:23:12.027 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:23:22.041 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:23:32.055 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:23:42.065 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:23:52.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:24:02.098 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:24:12.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:24:22.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:24:32.144 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:24:42.161 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:24:52.177 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:25:02.192 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:25:12.208 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:25:22.220 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:25:32.236 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:25:42.253 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-08T12:25:52.270 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:26:02.284 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:26:12.297 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:26:22.315 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:26:32.332 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-08T12:26:42.348 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_ofriedma@teuthology', 'description': '/home/teuthworker/archive/ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4581813', 'machine_type': 'smithi'} 2019-12-08T12:26:42.369 DEBUG:teuthology.lock.ops:locked ubuntu@smithi129.front.sepia.ceph.com, ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:26:42.420 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-08T12:26:42.604 INFO:teuthology.provision.fog.smithi129:Scheduling deploy of centos 7.6 2019-12-08T12:26:42.610 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 7.6 2019-12-08T12:26:42.923 INFO:teuthology.orchestra.console:Power off smithi181 2019-12-08T12:26:42.923 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-08T12:26:42.938 INFO:teuthology.orchestra.console:Power off smithi129 2019-12-08T12:26:42.939 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-08T12:26:42.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-08T12:26:42.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-08T12:26:43.074 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2019-12-08T12:26:43.077 INFO:teuthology.orchestra.console:Power off for smithi129 completed 2019-12-08T12:26:43.176 INFO:teuthology.orchestra.console:Power on smithi181 2019-12-08T12:26:43.176 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-08T12:26:43.185 INFO:teuthology.orchestra.console:Power on smithi129 2019-12-08T12:26:43.186 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-08T12:26:43.197 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-08T12:26:43.206 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-08T12:26:47.221 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-08T12:26:47.233 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-08T12:26:47.435 INFO:teuthology.orchestra.console:Power on for smithi181 completed 2019-12-08T12:26:47.449 INFO:teuthology.orchestra.console:Power on for smithi129 completed 2019-12-08T12:26:47.537 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2019-12-08T12:26:47.550 INFO:teuthology.provision.fog.smithi129:Waiting for deploy to finish 2019-12-08T12:29:18.736 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:18.739 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:27.822 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:27.823 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:36.878 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:36.879 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:45.933 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:45.934 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:54.989 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:29:54.990 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:30:04.046 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:30:04.047 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:30:13.099 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:30:13.100 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:19.102 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:19.104 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:19.293 INFO:teuthology.orchestra.run.smithi181:> hostname 2019-12-08T12:31:19.294 INFO:teuthology.orchestra.run.smithi129:> hostname 2019-12-08T12:31:19.859 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2019-12-08T12:31:19.860 INFO:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2019-12-08T12:31:19.870 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129 2019-12-08T12:31:19.870 INFO:teuthology.orchestra.run.smithi129:> grep smithi129 /etc/hosts 2019-12-08T12:31:19.938 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2019-12-08T12:31:19.939 INFO:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2019-12-08T12:31:19.958 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129 smithi129.front.sepia.ceph.com smithi129 2019-12-08T12:31:19.959 INFO:teuthology.orchestra.run.smithi129:> sudo hostname smithi129 2019-12-08T12:31:20.095 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2019-12-08T12:31:20.103 INFO:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/smithi129/smithi129/g' /etc/hosts 2019-12-08T12:31:20.200 INFO:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/172.21.15.129/172.21.15.129/g' /etc/hosts 2019-12-08T12:31:20.208 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2019-12-08T12:31:20.276 INFO:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/smithi129/smithi129/g' /etc/hostname 2019-12-08T12:31:20.295 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2019-12-08T12:31:20.345 INFO:teuthology.orchestra.run.smithi129:> sudo hostname smithi129 2019-12-08T12:31:20.377 INFO:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2019-12-08T12:31:20.405 INFO:teuthology.provision.fog.smithi129:Deploy complete! 2019-12-08T12:31:20.406 DEBUG:teuthology.parallel:result is None 2019-12-08T12:31:20.444 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2019-12-08T12:31:20.446 DEBUG:teuthology.parallel:result is None 2019-12-08T12:31:20.596 INFO:teuthology.lock.ops:Checking smithi129.front.sepia.ceph.com 2019-12-08T12:31:20.597 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-08T12:31:20.609 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2019-12-08T12:31:20.610 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-08T12:31:20.622 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: set os type and version on server 2019-12-08T12:31:20.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:31:20.623 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:20.713 INFO:teuthology.orchestra.run.smithi129:> true 2019-12-08T12:31:21.092 INFO:teuthology.orchestra.run.smithi129:> uname -m 2019-12-08T12:31:21.176 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2019-12-08T12:31:21.177 INFO:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2019-12-08T12:31:21.229 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Linux" 2019-12-08T12:31:21.229 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="7 (Core)" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="7" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-08T12:31:21.230 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.centos.org/" 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-08T12:31:21.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:21.232 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2019-12-08T12:31:21.250 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2019-12-08T12:31:21.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:31:21.250 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:21.341 INFO:teuthology.orchestra.run.smithi181:> true 2019-12-08T12:31:21.710 INFO:teuthology.orchestra.run.smithi181:> uname -m 2019-12-08T12:31:21.770 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2019-12-08T12:31:21.771 INFO:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2019-12-08T12:31:21.839 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Linux" 2019-12-08T12:31:21.839 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="7 (Core)" 2019-12-08T12:31:21.839 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2019-12-08T12:31:21.840 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2019-12-08T12:31:21.840 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="7" 2019-12-08T12:31:21.840 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-08T12:31:21.841 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2019-12-08T12:31:21.841 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-08T12:31:21.841 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.centos.org/" 2019-12-08T12:31:21.841 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-08T12:31:21.841 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:21.841 INFO:teuthology.orchestra.run.smithi181.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-08T12:31:21.842 INFO:teuthology.orchestra.run.smithi181.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-08T12:31:21.842 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-08T12:31:21.842 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-08T12:31:21.842 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:21.843 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2019-12-08T12:31:21.862 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-12-08T12:31:21.893 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi129.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK8tUMz9/A8JLSzurpLawRtI4Ag2KYFCFJYF1EhE0Dc6hrGCmge1QchfjO/h3Lu6ME/AuCB/Yqlh6DaPVTvX2SDYeHIlIjIquf5iSPU5CcU5+M7S5VNPnDPvMkb/IB08iQHEahu9j/bXyP9dms696Y83TIiOJMEKhrfdZBlDXUqE+2ixtbCL7a3C7qLRb9F6s/Ce/Jp0za1Q8e45sc05ypv2fQ4Den/OQdvrCpWkwbBFCBRXR8FPm80M3HKv02jEMbBNcsi7NI5OrTPdsn4b7nUyO/Jyb3ZHeoYxTOVbigs6GplcDjM8ZXosjor4IB+SWqCYVGxKxYuuSHAKxoK/6X smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCusmxpv7n8K5eMtzAHkq/cBvkn+DgwKs+Dg/Vm7O1R9EsgZPA18a/b9mkD37Y6pYpcM5o6pO1ulGPGKeiFtaZMlUyH4Nt5ywhEd6jsUg2gQMK2Lc0/L/CVwi7SeCuwOvWHRag9jzHl2mhfeVbWxGcniIQB6pCZ/tJ7QT1YM2j0IwOlDc3taQNDgxOc7oEdUAklKKva/O+huCvgygxHdcFKrke21d5t5Hq7sT+kuBtogao/QuC/wy5kzFbe0Z+NSE12l7k+NcoTyNp9aFJbvxTiFf0lVBHr+jVfoeofYTOfh6GjGYaFXfr8a4B8xwiqF9KNGjX8YtLfgEdLZwiwijwV 2019-12-08T12:31:21.893 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-08T12:31:21.937 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-12-08T12:31:21.944 INFO:teuthology.task.internal:Saving configuration 2019-12-08T12:31:21.967 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-12-08T12:31:21.974 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-12-08T12:31:21.989 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-08 12:26:42.354073', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi129.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEelMmBp6oWfyTIswLeLH+B/sB8X3yZ8+idOY391USroNJde6LIDyrSy7+jp/T00vRoYrOXFougVpKQfAA4wz1I=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4581813'} 2019-12-08T12:31:22.003 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-08 12:26:42.354092', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi181.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN77jyS6ndSSAsljcRU1cjvYVoU6REJsTNUCf/BGe/qn51+Ra21/rJFkIn4fTgbwmro4gbrkDrpj5Na9wJgwT0k=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4581813'} 2019-12-08T12:31:22.003 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-12-08T12:31:22.010 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-08T12:31:22.010 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-08T12:31:22.010 INFO:teuthology.run_tasks:Running task console_log... 2019-12-08T12:31:22.077 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-08T12:31:22.078 INFO:teuthology.run_tasks:Running task internal.connect... 2019-12-08T12:31:22.089 INFO:teuthology.task.internal:Opening connections... 2019-12-08T12:31:22.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:31:22.090 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:22.191 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:31:22.192 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:31:22.286 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-12-08T12:31:22.295 INFO:teuthology.orchestra.run.smithi129:> uname -m 2019-12-08T12:31:22.542 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2019-12-08T12:31:22.543 INFO:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2019-12-08T12:31:22.629 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Linux" 2019-12-08T12:31:22.629 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="7 (Core)" 2019-12-08T12:31:22.630 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2019-12-08T12:31:22.630 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2019-12-08T12:31:22.630 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="7" 2019-12-08T12:31:22.630 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-08T12:31:22.631 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2019-12-08T12:31:22.631 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-08T12:31:22.631 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.centos.org/" 2019-12-08T12:31:22.631 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-08T12:31:22.632 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:22.632 INFO:teuthology.orchestra.run.smithi129.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-08T12:31:22.632 INFO:teuthology.orchestra.run.smithi129.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-08T12:31:22.632 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-08T12:31:22.633 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-08T12:31:22.634 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:22.634 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2019-12-08T12:31:22.653 INFO:teuthology.orchestra.run.smithi181:> uname -m 2019-12-08T12:31:22.781 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2019-12-08T12:31:22.782 INFO:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2019-12-08T12:31:22.866 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Linux" 2019-12-08T12:31:22.866 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="7 (Core)" 2019-12-08T12:31:22.866 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2019-12-08T12:31:22.866 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2019-12-08T12:31:22.867 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="7" 2019-12-08T12:31:22.867 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-08T12:31:22.867 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2019-12-08T12:31:22.867 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-08T12:31:22.867 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.centos.org/" 2019-12-08T12:31:22.868 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-08T12:31:22.868 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:22.868 INFO:teuthology.orchestra.run.smithi181.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-08T12:31:22.868 INFO:teuthology.orchestra.run.smithi181.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-08T12:31:22.869 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-08T12:31:22.869 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-08T12:31:22.869 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:22.870 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2019-12-08T12:31:22.890 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-12-08T12:31:22.907 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-12-08T12:31:22.950 INFO:teuthology.task.internal:Checking for old test directory... 2019-12-08T12:31:22.951 INFO:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2019-12-08T12:31:22.955 INFO:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2019-12-08T12:31:23.046 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-12-08T12:31:23.060 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-12-08T12:31:23.060 INFO:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2019-12-08T12:31:23.065 INFO:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2019-12-08T12:31:23.164 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-12-08T12:31:23.206 INFO:teuthology.run_tasks:Running task kernel... 2019-12-08T12:31:23.254 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-12-08T12:31:23.254 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-12-08T12:31:23.254 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:31:23.255 INFO:teuthology.orchestra.run.smithi129:> sudo package-cleanup -y --oldkernels 2019-12-08T12:31:24.085 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:31:25.105 INFO:teuthology.orchestra.run.smithi129.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:25.106 INFO:teuthology.orchestra.run.smithi129.stdout:Trying other mirror. 2019-12-08T12:31:25.106 INFO:teuthology.orchestra.run.smithi129.stdout:To address this issue please refer to the below wiki article 2019-12-08T12:31:25.106 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:25.106 INFO:teuthology.orchestra.run.smithi129.stdout:https://wiki.centos.org/yum-errors 2019-12-08T12:31:25.106 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:25.106 INFO:teuthology.orchestra.run.smithi129.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-08T12:31:25.107 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:25.253 INFO:teuthology.orchestra.run.smithi129.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:25.254 INFO:teuthology.orchestra.run.smithi129.stdout:Trying other mirror. 2019-12-08T12:31:25.345 INFO:teuthology.orchestra.run.smithi129.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:25.346 INFO:teuthology.orchestra.run.smithi129.stdout:Trying other mirror. 2019-12-08T12:31:25.489 INFO:teuthology.orchestra.run.smithi129.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:25.489 INFO:teuthology.orchestra.run.smithi129.stdout:Trying other mirror. 2019-12-08T12:31:25.642 INFO:teuthology.orchestra.run.smithi129.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:25.642 INFO:teuthology.orchestra.run.smithi129.stdout:Trying other mirror. 2019-12-08T12:31:27.062 INFO:teuthology.orchestra.run.smithi129.stdout:No old kernels to remove 2019-12-08T12:31:27.078 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:31:27.078 INFO:teuthology.orchestra.run.smithi181:> sudo package-cleanup -y --oldkernels 2019-12-08T12:31:28.063 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:31:29.120 INFO:teuthology.orchestra.run.smithi181.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout:Trying other mirror. 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout:To address this issue please refer to the below wiki article 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout:https://wiki.centos.org/yum-errors 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-08T12:31:29.121 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:31:29.267 INFO:teuthology.orchestra.run.smithi181.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:29.267 INFO:teuthology.orchestra.run.smithi181.stdout:Trying other mirror. 2019-12-08T12:31:29.359 INFO:teuthology.orchestra.run.smithi181.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:29.359 INFO:teuthology.orchestra.run.smithi181.stdout:Trying other mirror. 2019-12-08T12:31:29.502 INFO:teuthology.orchestra.run.smithi181.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:29.502 INFO:teuthology.orchestra.run.smithi181.stdout:Trying other mirror. 2019-12-08T12:31:29.655 INFO:teuthology.orchestra.run.smithi181.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-08T12:31:29.655 INFO:teuthology.orchestra.run.smithi181.stdout:Trying other mirror. 2019-12-08T12:31:31.288 INFO:teuthology.orchestra.run.smithi181.stdout:No old kernels to remove 2019-12-08T12:31:31.308 INFO:teuthology.orchestra.run.smithi129:> uname -r 2019-12-08T12:31:31.404 INFO:teuthology.orchestra.run.smithi129.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-08T12:31:31.405 INFO:teuthology.task.kernel:Running kernel on smithi129: 3.10.0-1062.4.1.el7.x86_64 2019-12-08T12:31:31.405 INFO:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2019-12-08T12:31:31.628 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:31:31.664 INFO:teuthology.orchestra.run.smithi129.stdout:Determining fastest mirrors 2019-12-08T12:31:31.822 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:31:31.823 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: ftp.linux.ncsu.edu 2019-12-08T12:31:31.823 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.wdc1.us.leaseweb.net 2019-12-08T12:31:31.824 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:31:33.257 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:31:33.258 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:31:33.258 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-08T12:31:34.656 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:35.525 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:31:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-08T12:31:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 50 M 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 64 M 2019-12-08T12:31:35.528 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:31:35.532 INFO:teuthology.orchestra.run.smithi129.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-08T12:31:36.369 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:31:36.548 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:31:37.499 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:31:37.500 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:32:12.111 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-08T12:32:12.657 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-08T12:32:12.657 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:32:12.658 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:32:12.658 INFO:teuthology.orchestra.run.smithi129.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-08T12:32:12.658 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:32:12.658 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:32:12.726 INFO:teuthology.orchestra.run.smithi129:> rpm -q kernel --last | head -n 1 2019-12-08T12:32:13.131 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 08 Dec 2019 12:31:40 PM UTC 2019-12-08T12:32:13.133 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:13.133 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:13.133 INFO:teuthology.orchestra.run.smithi181:> uname -r 2019-12-08T12:32:13.252 INFO:teuthology.orchestra.run.smithi181.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-08T12:32:13.253 INFO:teuthology.task.kernel:Running kernel on smithi181: 3.10.0-1062.4.1.el7.x86_64 2019-12-08T12:32:13.253 INFO:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2019-12-08T12:32:13.525 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:32:13.568 INFO:teuthology.orchestra.run.smithi181.stdout:Determining fastest mirrors 2019-12-08T12:32:13.731 INFO:teuthology.orchestra.run.smithi181.stdout: * base: ftp.osuosl.org 2019-12-08T12:32:13.731 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: ftp.linux.ncsu.edu 2019-12-08T12:32:13.732 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.wdc1.us.leaseweb.net 2019-12-08T12:32:13.732 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:32:15.367 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:32:15.368 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:32:15.368 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-08T12:32:16.708 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:32:17.570 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:17.571 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:32:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:17.580 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:32:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:32:17.581 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:32:17.581 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:32:17.581 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-08T12:32:17.581 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:17.581 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:32:17.581 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:32:17.582 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2019-12-08T12:32:17.582 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:17.582 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 50 M 2019-12-08T12:32:17.582 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 64 M 2019-12-08T12:32:17.582 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:32:17.588 INFO:teuthology.orchestra.run.smithi181.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-08T12:32:18.956 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:32:19.174 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:32:20.103 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:32:20.104 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:32:54.773 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-08T12:32:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-08T12:32:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:55.348 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:32:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-08T12:32:55.349 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:55.349 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:32:55.397 INFO:teuthology.orchestra.run.smithi181:> rpm -q kernel --last | head -n 1 2019-12-08T12:32:55.684 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 08 Dec 2019 12:32:22 PM UTC 2019-12-08T12:32:55.686 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:55.686 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:55.686 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-12-08T12:32:55.686 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-08T12:32:55.687 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-08T12:32:55.687 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-12-08T12:32:55.687 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2019-12-08T12:32:55.687 INFO:teuthology.orchestra.run.smithi129:> rpm -q kernel --last | head -n 1 2019-12-08T12:32:56.055 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 08 Dec 2019 12:31:40 PM UTC 2019-12-08T12:32:56.056 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:56.056 INFO:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2 2019-12-08T12:32:56.218 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2 2019-12-08T12:32:56.218 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2019-12-08T12:32:56.218 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.02 2019-12-08T12:32:56.218 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 0.76.el7.centos 2019-12-08T12:32:56.218 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2019-12-08T12:32:56.218 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-08T12:32:56.219 INFO:teuthology.orchestra.run.smithi129.stdout:Group : System Environment/Base 2019-12-08T12:32:56.219 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 0 2019-12-08T12:32:56.219 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2019-12-08T12:32:56.219 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-08T12:32:56.219 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-08T12:32:56.219 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:Relocations : (not relocatable) 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : CentOS BuildSystem 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-08T12:32:56.220 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2019-12-08T12:32:56.221 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:32:56.221 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-08T12:32:56.221 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-08T12:32:56.221 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-08T12:32:56.221 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2019-12-08T12:32:56.221 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-08T12:32:56.222 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:56.222 INFO:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-08T12:32:56.714 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2019-12-08T12:32:56.855 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:56.870 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-08T12:32:57.147 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-08T12:32:57.193 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-08T12:32:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-08T12:32:57.263 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-08T12:32:57.300 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-08T12:32:57.312 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-08T12:32:58.484 INFO:teuthology.orchestra.run.smithi129.stderr:done 2019-12-08T12:32:58.486 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:32:58.540 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.ibbGr93RUT 2019-12-08T12:32:58.541 INFO:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ibbGr93RUT 2019-12-08T12:32:58.601 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.ibbGr93RUT 2019-12-08T12:32:58.743 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.ibbGr93RUT is 6KB 2019-12-08T12:32:58.799 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.ibbGr93RUT 2019-12-08T12:32:58.852 INFO:teuthology.orchestra.run.smithi129:> sudo grub2-set-default 0 2019-12-08T12:32:59.096 INFO:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2019-12-08T12:32:59.099 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-12-08T12:32:59.099 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=None, version=distro) 2019-12-08T12:32:59.100 INFO:teuthology.orchestra.run.smithi181:> rpm -q kernel --last | head -n 1 2019-12-08T12:32:59.460 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 08 Dec 2019 12:32:22 PM UTC 2019-12-08T12:32:59.462 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:59.462 INFO:teuthology.orchestra.run.smithi181:> sudo rpm -qi grub2 2019-12-08T12:32:59.582 INFO:teuthology.orchestra.run.smithi181.stdout:Name : grub2 2019-12-08T12:32:59.582 INFO:teuthology.orchestra.run.smithi181.stdout:Epoch : 1 2019-12-08T12:32:59.582 INFO:teuthology.orchestra.run.smithi181.stdout:Version : 2.02 2019-12-08T12:32:59.582 INFO:teuthology.orchestra.run.smithi181.stdout:Release : 0.76.el7.centos 2019-12-08T12:32:59.582 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: x86_64 2019-12-08T12:32:59.582 INFO:teuthology.orchestra.run.smithi181.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-08T12:32:59.583 INFO:teuthology.orchestra.run.smithi181.stdout:Group : System Environment/Base 2019-12-08T12:32:59.583 INFO:teuthology.orchestra.run.smithi181.stdout:Size : 0 2019-12-08T12:32:59.583 INFO:teuthology.orchestra.run.smithi181.stdout:License : GPLv3+ 2019-12-08T12:32:59.583 INFO:teuthology.orchestra.run.smithi181.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-08T12:32:59.583 INFO:teuthology.orchestra.run.smithi181.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-08T12:32:59.583 INFO:teuthology.orchestra.run.smithi181.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:Relocations : (not relocatable) 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:Packager : CentOS BuildSystem 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:Vendor : CentOS 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-08T12:32:59.584 INFO:teuthology.orchestra.run.smithi181.stdout:Description : 2019-12-08T12:32:59.585 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:32:59.585 INFO:teuthology.orchestra.run.smithi181.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-08T12:32:59.585 INFO:teuthology.orchestra.run.smithi181.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-08T12:32:59.585 INFO:teuthology.orchestra.run.smithi181.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-08T12:32:59.585 INFO:teuthology.orchestra.run.smithi181.stdout:hardware devices. 2019-12-08T12:32:59.586 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-08T12:32:59.586 INFO:teuthology.task.kernel:Updating grub on smithi181 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:32:59.586 INFO:teuthology.orchestra.run.smithi181:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-08T12:33:00.039 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2019-12-08T12:33:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:33:00.197 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-08T12:33:00.509 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-08T12:33:00.535 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-08T12:33:00.577 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-08T12:33:00.593 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-08T12:33:00.626 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-08T12:33:00.637 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-08T12:33:01.704 INFO:teuthology.orchestra.run.smithi181.stderr:done 2019-12-08T12:33:01.704 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:33:01.759 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.QBNiNQE4lU 2019-12-08T12:33:01.759 INFO:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QBNiNQE4lU 2019-12-08T12:33:01.822 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.QBNiNQE4lU 2019-12-08T12:33:01.947 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.QBNiNQE4lU is 6KB 2019-12-08T12:33:01.997 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.QBNiNQE4lU 2019-12-08T12:33:02.041 INFO:teuthology.orchestra.run.smithi181:> sudo grub2-set-default 0 2019-12-08T12:33:02.187 INFO:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2019-12-08T12:33:02.190 INFO:teuthology.misc:Re-opening connections... 2019-12-08T12:33:02.191 INFO:teuthology.misc:trying to connect to ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:33:02.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:33:02.191 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:34:02.193 DEBUG:teuthology.orchestra.remote:timed out 2019-12-08T12:34:02.193 INFO:teuthology.misc:trying to connect to ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:34:02.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:34:02.194 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:35:02.195 DEBUG:teuthology.orchestra.remote:timed out 2019-12-08T12:35:02.195 DEBUG:teuthology.misc:waited 120.004506826 2019-12-08T12:35:03.196 INFO:teuthology.misc:trying to connect to ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:35:03.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:35:03.197 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:35:03.298 INFO:teuthology.orchestra.run.smithi129:> true 2019-12-08T12:35:03.740 DEBUG:teuthology.misc:waited 121.548932791 2019-12-08T12:35:04.741 INFO:teuthology.misc:trying to connect to ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:35:04.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:35:04.742 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:35:04.849 INFO:teuthology.orchestra.run.smithi181:> true 2019-12-08T12:35:05.299 DEBUG:teuthology.misc:waited 123.108200788 2019-12-08T12:35:06.299 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-12-08T12:35:06.300 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-08T12:35:06.300 INFO:teuthology.orchestra.run.smithi129:> uname -r 2019-12-08T12:35:06.405 INFO:teuthology.orchestra.run.smithi129.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:35:06.406 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:35:06.406 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-08T12:35:06.406 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-08T12:35:06.406 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-08T12:35:06.406 INFO:teuthology.orchestra.run.smithi181:> uname -r 2019-12-08T12:35:06.505 INFO:teuthology.orchestra.run.smithi181.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:35:06.506 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-08T12:35:06.506 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-08T12:35:07.507 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-12-08T12:35:07.507 INFO:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-08T12:35:07.648 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2019-12-08T12:35:07.661 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-12-08T12:35:07.661 INFO:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-08T12:35:07.835 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2019-12-08T12:35:07.847 INFO:teuthology.run_tasks:Running task internal.base... 2019-12-08T12:35:07.857 INFO:teuthology.task.internal:Creating test directory... 2019-12-08T12:35:07.857 INFO:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-08T12:35:07.861 INFO:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-08T12:35:07.952 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-12-08T12:35:07.964 INFO:teuthology.run_tasks:Running task internal.archive... 2019-12-08T12:35:08.003 INFO:teuthology.task.internal:Creating archive directory... 2019-12-08T12:35:08.004 INFO:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-08T12:35:08.007 INFO:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-08T12:35:08.111 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-12-08T12:35:08.146 INFO:teuthology.task.internal:Enabling coredump saving... 2019-12-08T12:35:08.146 INFO:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-08T12:35:08.166 INFO:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-08T12:35:08.284 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-08T12:35:08.289 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-08T12:35:08.291 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-12-08T12:35:08.302 INFO:teuthology.task.internal:Configuring sudo... 2019-12-08T12:35:08.302 INFO:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-08T12:35:08.308 INFO:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-08T12:35:08.413 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-12-08T12:35:08.427 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-12-08T12:35:08.427 INFO:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-08T12:35:08.431 INFO:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-08T12:35:08.519 INFO:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-08T12:35:08.601 INFO:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-08T12:35:08.686 INFO:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-08T12:35:08.761 INFO:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-08T12:35:08.838 INFO:teuthology.orchestra.run.smithi129:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-08T12:35:08.901 INFO:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-08T12:35:09.011 INFO:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-08T12:35:09.146 INFO:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-08T12:35:09.240 INFO:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-08T12:35:09.334 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-08T12:35:09.410 INFO:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2019-12-08T12:35:09.414 INFO:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2019-12-08T12:35:09.570 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-08T12:35:09.578 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-08T12:35:09.656 INFO:teuthology.run_tasks:Running task internal.timer... 2019-12-08T12:35:09.667 INFO:teuthology.task.internal:Starting timer... 2019-12-08T12:35:09.668 INFO:teuthology.run_tasks:Running task pcp... 2019-12-08T12:35:09.708 INFO:teuthology.run_tasks:Running task selinux... 2019-12-08T12:35:09.720 INFO:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2019-12-08T12:35:09.883 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: [ OK ] 2019-12-08T12:35:09.887 INFO:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2019-12-08T12:35:10.052 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: [ OK ] 2019-12-08T12:35:10.055 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-12-08T12:35:10.055 INFO:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2019-12-08T12:35:10.152 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2019-12-08T12:35:10.152 INFO:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2019-12-08T12:35:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2019-12-08T12:35:10.242 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2019-12-08T12:35:10.242 INFO:teuthology.orchestra.run.smithi129:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-08T12:35:10.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-08T12:35:10.360 INFO:teuthology.orchestra.run.smithi181:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-08T12:35:10.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-08T12:35:10.480 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-12-08T12:35:10.480 INFO:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2019-12-08T12:35:10.615 INFO:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2019-12-08T12:35:10.757 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-12-08T12:35:10.770 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-12-08T12:35:10.770 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-12-08T12:35:10.785 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-12-08T12:35:10.788 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-12-08T12:35:11.183 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-12-08T12:35:12.353 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2019-12-08T12:35:15.014 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-12-08T12:35:15.038 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:15 +0000 (0:00:00.068) 0:00:00.068 ******* 2019-12-08T12:35:15.050 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:15 +0000 (0:00:00.012) 0:00:00.081 ******* 2019-12-08T12:35:18.322 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi181.front.sepia.ceph.com] 2019-12-08T12:35:18.337 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] 2019-12-08T12:35:18.441 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:18 +0000 (0:00:03.390) 0:00:03.471 ******* 2019-12-08T12:35:18.476 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:18 +0000 (0:00:00.034) 0:00:03.506 ******* 2019-12-08T12:35:18.876 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-12-08T12:35:18.878 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-08T12:35:18.924 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:18 +0000 (0:00:00.447) 0:00:03.954 ******* 2019-12-08T12:35:18.930 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-08T12:35:18.974 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:18 +0000 (0:00:00.050) 0:00:04.005 ******* 2019-12-08T12:35:19.494 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-12-08T12:35:19.496 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-08T12:35:19.534 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:19 +0000 (0:00:00.559) 0:00:04.565 ******* 2019-12-08T12:35:19.540 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-08T12:35:19.577 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:19 +0000 (0:00:00.042) 0:00:04.607 ******* 2019-12-08T12:35:19.587 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-12-08T12:35:19.588 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:19.624 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:19 +0000 (0:00:00.047) 0:00:04.655 ******* 2019-12-08T12:35:19.631 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:19.667 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:19 +0000 (0:00:00.042) 0:00:04.698 ******* 2019-12-08T12:35:20.115 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-12-08T12:35:20.117 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:35:20.152 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:20 +0000 (0:00:00.485) 0:00:05.183 ******* 2019-12-08T12:35:20.159 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:35:20.201 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:20 +0000 (0:00:00.048) 0:00:05.231 ******* 2019-12-08T12:35:21.536 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-12-08T12:35:21.537 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808520.17-214407454484622/source", "state": "file", "uid": 0} 2019-12-08T12:35:21.575 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:21 +0000 (0:00:01.374) 0:00:06.606 ******* 2019-12-08T12:35:21.583 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808520.22-169772794981007/source", "state": "file", "uid": 0} 2019-12-08T12:35:21.648 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:21 +0000 (0:00:00.070) 0:00:06.678 ******* 2019-12-08T12:35:22.148 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:22.181 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:22.504 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:22.515 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:22.828 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:22.861 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:23.168 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:23.200 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:23.504 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "change 2019-12-08T12:35:23.506 INFO:teuthology.task.ansible.out:d": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:23.821 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:23.843 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:24.157 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:24.209 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-08T12:35:24.337 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2019-12-08T12:35:24.342 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-12-08T12:35:24.531 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:24 +0000 (0:00:02.883) 0:00:09.562 ******* 2019-12-08T12:35:24.564 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:24 +0000 (0:00:00.032) 0:00:09.595 ******* 2019-12-08T12:35:25.483 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi129.front.sepia.ceph.com] 2019-12-08T12:35:25.531 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] 2019-12-08T12:35:25.584 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-12-08T12:35:25.586 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:25 +0000 (0:00:01.020) 0:00:10.615 ******* 2019-12-08T12:35:25.659 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:25 +0000 (0:00:00.074) 0:00:10.689 ******* 2019-12-08T12:35:26.091 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-12-08T12:35:26.092 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.235608", "end": "2019-12-08 12:35:26.069368", "rc": 0, "start": "2019-12-08 12:35:25.833760", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-08T12:35:26.126 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.467) 0:00:11.157 ******* 2019-12-08T12:35:26.131 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.243692", "end": "2019-12-08 12:35:26.083947", "rc": 0, "start": "2019-12-08 12:35:25.840255", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-08T12:35:26.169 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.042) 0:00:11.199 ******* 2019-12-08T12:35:26.483 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-08T12:35:26.484 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.020108", "end": "2019-12-08 12:35:26.449105", "failed_when_result": false, "rc": 0, "start": "2019-12-08 12:35:26.428997", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-08T12:35:26.522 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.353) 0:00:11.553 ******* 2019-12-08T12:35:26.549 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-08T12:35:26.550 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:26.587 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.064) 0:00:11.617 ******* 2019-12-08T12:35:26.612 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-08T12:35:26.613 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:26.651 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.063) 0:00:11.681 ******* 2019-12-08T12:35:26.677 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-08T12:35:26.678 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:26.716 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.064) 0:00:11.746 ******* 2019-12-08T12:35:26.742 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-08T12:35:26.743 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:26.782 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.066) 0:00:11.813 ******* 2019-12-08T12:35:26.808 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-08T12:35:26.810 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:26.847 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:26 +0000 (0:00:00.064) 0:00:11.877 ******* 2019-12-08T12:35:27.169 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-12-08T12:35:27.170 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:35:27.208 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.361) 0:00:12.238 ******* 2019-12-08T12:35:27.525 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-08T12:35:27.526 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:01.014233", "end": "2019-12-08 12:35:27.495493", "failed_when_result": false, "rc": 0, "start": "2019-12-08 12:35:26.481260", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-08T12:35:27.581 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.373) 0:00:12.612 ******* 2019-12-08T12:35:27.593 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-08T12:35:27.593 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-08T12:35:27.616 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-08T12:35:27.618 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:27.657 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.075) 0:00:12.687 ******* 2019-12-08T12:35:27.683 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-08T12:35:27.684 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:27.747 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.076) 0:00:12.764 ******* 2019-12-08T12:35:27.784 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-08T12:35:27.786 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:27.823 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.089) 0:00:12.853 ******* 2019-12-08T12:35:27.849 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-08T12:35:27.850 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:27.889 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.066) 0:00:12.919 ******* 2019-12-08T12:35:27.915 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-08T12:35:27.916 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-08T12:35:27.917 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:35:27.955 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:27 +0000 (0:00:00.065) 0:00:12.985 ******* 2019-12-08T12:35:28.250 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-08T12:35:28.250 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-08T12:35:28.285 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-12-08T12:35:28.286 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:35:28.347 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:35:28 +0000 (0:00:00.392) 0:00:13.378 ******* 2019-12-08T12:35:28.584 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-08T12:35:28.588 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-08T12:35:28.760 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-08T12:35:28.918 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-08T12:35:29.098 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-08T12:35:29.247 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-08T12:35:29.440 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-08T12:35:29.595 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-08T12:35:29.799 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-08T12:35:29.924 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-08T12:35:30.139 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-08T12:35:30.265 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-08T12:35:30.494 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-08T12:35:30.610 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-08T12:35:30.836 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-08T12:35:30.917 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-08T12:35:31.157 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-08T12:35:31.294 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-08T12:35:31.504 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-08T12:35:31.597 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-08T12:35:31.817 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-08T12:35:31.923 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-08T12:35:32.151 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-08T12:35:32.245 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-08T12:35:32.497 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-08T12:35:32.572 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-08T12:35:32.842 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-08T12:35:32.908 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-08T12:35:33.173 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-08T12:35:33.237 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-08T12:35:33.511 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-08T12:35:33.575 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-08T12:35:33.843 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-08T12:35:33.916 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-08T12:35:34.188 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-08T12:35:34.242 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-08T12:35:34.536 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-08T12:35:34.569 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-08T12:35:34.898 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-08T12:35:34.915 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-08T12:35:35.218 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-08T12:35:35.255 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-08T12:35:35.532 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-08T12:35:35.585 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-08T12:35:35.859 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-08T12:35:35.934 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-08T12:35:36.211 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-08T12:35:36.281 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-08T12:35:36.530 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-08T12:35:36.599 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-08T12:35:36.846 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-08T12:35:36.940 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-08T12:35:37.194 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-08T12:35:37.289 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-08T12:35:37.545 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-08T12:35:37.632 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-08T12:35:37.867 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-08T12:35:37.952 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-08T12:35:38.207 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-08T12:35:38.308 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-08T12:35:38.550 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-08T12:35:38.656 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-08T12:35:38.855 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-08T12:35:38.993 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-08T12:35:39.200 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-08T12:35:39.312 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-08T12:35:39.520 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-08T12:35:39.639 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-08T12:35:39.879 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-08T12:35:39.973 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-08T12:35:40.217 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-08T12:35:40.336 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-08T12:35:40.573 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-08T12:35:40.682 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-08T12:35:40.926 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-08T12:35:41.027 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-08T12:35:41.284 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-08T12:35:41.285 INFO:teuthology.task.ansible.out: 2019-12-08T12:35:41.359 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-08T12:35:41.611 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-08T12:35:41.691 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-08T12:35:41.953 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-08T12:35:42.020 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-08T12:35:42.275 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-08T12:35:42.371 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-08T12:35:42.601 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-08T12:35:42.722 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-08T12:35:42.944 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-08T12:35:43.062 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-08T12:35:43.246 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-08T12:35:43.351 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-08T12:35:43.554 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-08T12:35:43.700 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-08T12:35:43.873 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-08T12:35:44.051 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-08T12:35:44.221 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-08T12:35:44.388 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-08T12:35:44.567 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-08T12:35:44.725 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-08T12:35:44.935 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-08T12:35:45.089 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-08T12:35:45.301 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-08T12:35:45.452 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-08T12:35:45.645 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-08T12:35:45.809 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-08T12:35:45.974 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-08T12:35:46.162 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-08T12:35:46.320 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-08T12:35:46.508 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-08T12:35:46.654 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-08T12:35:46.830 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-08T12:35:46.991 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-08T12:35:47.185 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-08T12:35:47.334 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-08T12:35:47.516 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-08T12:35:47.670 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-08T12:35:47.839 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-08T12:35:47.975 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-08T12:35:48.163 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-08T12:35:48.335 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-08T12:35:48.502 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-08T12:35:48.654 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-08T12:35:48.820 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-08T12:35:48.995 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-08T12:35:49.143 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-08T12:35:49.345 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-08T12:35:49.485 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-08T12:35:49.664 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-08T12:35:49.834 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-08T12:35:50.017 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-08T12:35:50.171 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-08T12:35:50.369 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-08T12:35:50.498 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-08T12:35:50.700 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-08T12:35:50.838 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-08T12:35:51.027 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-08T12:35:51.184 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-08T12:35:51.384 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-08T12:35:51.531 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-08T12:35:51.712 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-08T12:35:51.877 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-08T12:35:52.013 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-08T12:35:52.213 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-08T12:35:52.356 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-08T12:35:52.545 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-08T12:35:52.665 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-08T12:35:52.884 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-08T12:35:52.999 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-08T12:35:53.218 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-08T12:35:53.334 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-08T12:35:53.544 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-08T12:35:53.652 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-08T12:35:53.855 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-08T12:35:53.972 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-08T12:35:54.184 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-08T12:35:54.281 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-08T12:35:54.509 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-08T12:35:54.612 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-08T12:35:54.831 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-08T12:35:54.933 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-08T12:35:55.169 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-08T12:35:55.270 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-08T12:35:55.501 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-08T12:35:55.607 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-08T12:35:55.841 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-08T12:35:55.929 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-08T12:35:56.166 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-08T12:35:56.272 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-08T12:35:56.496 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-08T12:35:56.623 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-08T12:35:56.815 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-08T12:35:56.941 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-08T12:35:57.145 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-08T12:35:57.283 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-08T12:35:57.496 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-08T12:35:57.628 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-08T12:35:57.823 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-08T12:35:57.980 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-08T12:35:58.148 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-08T12:35:58.314 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-08T12:35:58.482 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-08T12:35:58.657 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-08T12:35:58.839 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-08T12:35:59.023 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-08T12:35:59.203 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-08T12:35:59.383 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-08T12:35:59.538 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-08T12:35:59.729 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-08T12:35:59.860 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-08T12:36:00.060 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-08T12:36:00.188 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-08T12:36:00.391 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-08T12:36:00.522 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-08T12:36:00.756 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-08T12:36:00.860 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-08T12:36:01.114 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-08T12:36:01.209 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-08T12:36:01.472 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-08T12:36:01.536 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-08T12:36:01.848 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-08T12:36:01.860 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-08T12:36:02.188 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-08T12:36:02.200 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-08T12:36:02.530 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-08T12:36:02.824 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-08T12:36:02.958 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-08T12:36:03.000 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:02 +0000 (0:00:34.652) 0:00:48.030 ******* 2019-12-08T12:36:03.026 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-08T12:36:03.073 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:03 +0000 (0:00:00.073) 0:00:48.104 ******* 2019-12-08T12:36:03.123 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-08T12:36:03.127 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-08T12:36:03.128 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-08T12:36:03.130 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-08T12:36:03.131 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-08T12:36:03.133 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJy 2019-12-08T12:36:03.139 INFO:teuthology.task.ansible.out:w 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 2019-12-08T12:36:03.140 INFO:teuthology.task.ansible.out: 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf853 2019-12-08T12:36:03.142 INFO:teuthology.task.ansible.out:2237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36b 2019-12-08T12:36:03.143 INFO:teuthology.task.ansible.out:deb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a28 2019-12-08T12:36:03.145 INFO:teuthology.task.ansible.out:73481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c 2019-12-08T12:36:03.146 INFO:teuthology.task.ansible.out:4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-08T12:36:03.193 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:03 +0000 (0:00:00.119) 0:00:48.223 ******* 2019-12-08T12:36:03.220 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-08T12:36:03.221 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-08T12:36:03.566 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-08T12:36:04.168 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-08T12:36:04.293 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-08T12:36:04.294 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-08T12:36:04.330 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:04 +0000 (0:00:01.137) 0:00:49.361 ******* 2019-12-08T12:36:04.367 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:04 +0000 (0:00:00.036) 0:00:49.398 ******* 2019-12-08T12:36:04.373 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-08T12:36:04.418 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:04 +0000 (0:00:00.050) 0:00:49.449 ******* 2019-12-08T12:36:04.477 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-08T12:36:04.481 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-08T12:36:04.483 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-08T12:36:04.484 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-08T12:36:04.486 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-08T12:36:04.487 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-08T12:36:04.489 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-08T12:36:04.491 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 2019-12-08T12:36:04.492 INFO:teuthology.task.ansible.out:6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f8 2019-12-08T12:36:04.494 INFO:teuthology.task.ansible.out:48846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8a 2019-12-08T12:36:04.495 INFO:teuthology.task.ansible.out:a01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalho 2019-12-08T12:36:04.496 INFO:teuthology.task.ansible.out:jr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-08T12:36:04.537 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:04 +0000 (0:00:00.118) 0:00:49.568 ******* 2019-12-08T12:36:04.755 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-08T12:36:04.757 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-08T12:36:05.112 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-08T12:36:05.299 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi129.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-08T12:36:05.333 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:05 +0000 (0:00:00.796) 0:00:50.364 ******* 2019-12-08T12:36:05.465 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-08T12:36:05.467 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-08T12:36:05.723 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-08T12:36:05.808 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-08T12:36:06.043 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-08T12:36:06.150 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-08T12:36:06.152 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-08T12:36:06.368 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-08T12:36:06.497 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-08T12:36:06.708 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-08T12:36:06.835 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-08T12:36:06.837 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-08T12:36:07.002 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-08T12:36:07.004 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-08T12:36:07.191 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-08T12:36:07.357 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-08T12:36:07.514 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-08T12:36:07.525 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:07.702 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-08T12:36:07.703 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-08T12:36:07.869 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-08T12:36:07.870 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-08T12:36:07.872 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:07.882 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:07.893 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:08.020 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-08T12:36:08.233 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-08T12:36:08.390 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-08T12:36:08.432 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:08.549 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-08T12:36:08.550 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-08T12:36:08.779 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-08T12:36:08.781 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-08T12:36:08.791 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:08.802 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:08.813 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:08.898 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-08T12:36:09.140 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-08T12:36:09.256 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-08T12:36:09.487 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-08T12:36:09.488 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-08T12:36:09.598 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-08T12:36:09.845 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-08T12:36:09.951 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-08T12:36:10.203 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-08T12:36:10.292 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-08T12:36:10.526 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-08T12:36:10.611 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-08T12:36:10.884 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-08T12:36:10.980 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-08T12:36:10.981 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-08T12:36:11.228 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-08T12:36:11.333 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-08T12:36:11.335 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-08T12:36:11.559 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-08T12:36:11.690 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-08T12:36:11.900 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-08T12:36:11.901 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-08T12:36:12.026 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-08T12:36:12.256 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-08T12:36:12.258 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-08T12:36:12.372 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-08T12:36:12.374 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-08T12:36:12.586 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-08T12:36:12.728 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-08T12:36:12.909 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-08T12:36:13.078 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-08T12:36:13.261 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-08T12:36:13.263 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-08T12:36:13.430 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-08T12:36:13.619 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-08T12:36:13.764 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-08T12:36:13.974 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-08T12:36:14.100 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-08T12:36:14.330 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-08T12:36:14.444 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-08T12:36:14.446 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-08T12:36:14.663 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-08T12:36:14.791 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-08T12:36:15.022 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-08T12:36:15.139 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-08T12:36:15.392 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-08T12:36:15.393 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-08T12:36:15.468 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-08T12:36:15.742 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-08T12:36:15.828 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-08T12:36:16.081 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-08T12:36:16.157 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-08T12:36:16.420 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-08T12:36:16.463 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-08T12:36:16.764 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-08T12:36:16.777 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-08T12:36:17.111 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-08T12:36:17.154 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-08T12:36:17.447 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-08T12:36:17.502 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-08T12:36:17.802 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57 2019-12-08T12:36:17.804 INFO:teuthology.task.ansible.out:cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-08T12:36:18.134 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-08T12:36:18.156 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-08T12:36:18.158 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-08T12:36:18.481 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-08T12:36:18.506 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-08T12:36:18.847 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-08T12:36:18.853 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-08T12:36:18.871 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-08T12:36:19.210 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-08T12:36:19.212 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-08T12:36:19.546 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-08T12:36:19.560 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-08T12:36:19.561 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-08T12:36:19.879 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-08T12:36:19.933 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-08T12:36:19.934 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-08T12:36:19.936 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-08T12:36:20.234 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-08T12:36:20.256 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-08T12:36:20.580 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-08T12:36:20.582 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-08T12:36:20.592 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-08T12:36:20.951 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-08T12:36:20.963 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-08T12:36:20.964 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-08T12:36:20.966 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-08T12:36:21.322 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-08T12:36:21.344 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-08T12:36:21.712 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-08T12:36:21.714 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-08T12:36:22.062 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-08T12:36:22.073 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-08T12:36:22.419 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-08T12:36:22.452 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-08T12:36:22.773 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-08T12:36:22.805 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-08T12:36:23.139 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-08T12:36:23.140 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-08T12:36:23.141 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-08T12:36:23.511 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-08T12:36:23.523 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-08T12:36:23.864 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-08T12:36:23.875 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-08T12:36:24.211 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-08T12:36:24.213 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-08T12:36:24.244 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-08T12:36:24.245 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-08T12:36:24.547 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-08T12:36:24.611 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-08T12:36:24.873 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-08T12:36:24.970 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-08T12:36:25.236 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-08T12:36:25.238 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-08T12:36:25.324 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-08T12:36:25.593 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-08T12:36:25.667 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-08T12:36:25.921 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-08T12:36:25.999 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-08T12:36:26.283 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-08T12:36:26.368 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-08T12:36:26.632 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-08T12:36:26.727 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-08T12:36:26.962 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-08T12:36:27.061 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-08T12:36:27.298 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-08T12:36:27.382 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-08T12:36:27.645 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-08T12:36:27.741 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-08T12:36:27.969 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-08T12:36:28.043 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-08T12:36:28.285 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-08T12:36:28.328 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-08T12:36:28.634 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-08T12:36:28.634 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-08T12:36:28.952 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-08T12:36:28.973 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-08T12:36:28.975 INFO:teuthology.task.ansible.out: 2019-12-08T12:36:29.241 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-08T12:36:29.303 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-08T12:36:29.609 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-08T12:36:29.673 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-08T12:36:29.959 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-08T12:36:30.014 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-08T12:36:30.283 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-08T12:36:30.400 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-08T12:36:30.632 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-08T12:36:30.757 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-08T12:36:30.945 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-08T12:36:31.114 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-08T12:36:31.294 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-08T12:36:31.465 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-08T12:36:31.646 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-08T12:36:31.798 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-08T12:36:31.966 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-08T12:36:32.145 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-08T12:36:32.309 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-08T12:36:32.460 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-08T12:36:32.674 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-08T12:36:32.801 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-08T12:36:33.040 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-08T12:36:33.042 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-08T12:36:33.146 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-08T12:36:33.371 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-08T12:36:33.372 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-08T12:36:33.508 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-08T12:36:33.779 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-08T12:36:33.844 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-08T12:36:34.107 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-08T12:36:34.199 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-08T12:36:34.200 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-08T12:36:34.475 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-08T12:36:34.549 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-08T12:36:34.551 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-08T12:36:34.845 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-08T12:36:34.928 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-08T12:36:35.187 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-08T12:36:35.241 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-08T12:36:35.543 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-08T12:36:35.565 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-08T12:36:35.862 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-08T12:36:35.917 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-08T12:36:36.213 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-08T12:36:36.267 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-08T12:36:36.540 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-08T12:36:36.606 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-08T12:36:36.850 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-08T12:36:36.968 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-08T12:36:37.212 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-08T12:36:37.311 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-08T12:36:37.550 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-08T12:36:37.666 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-08T12:36:37.883 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-08T12:36:38.012 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-08T12:36:38.235 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-08T12:36:38.374 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-08T12:36:38.568 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-08T12:36:38.570 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-08T12:36:38.733 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-08T12:36:38.895 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-08T12:36:39.080 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-08T12:36:39.217 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-08T12:36:39.330 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:39 +0000 (0:00:33.997) 0:01:24.361 ******* 2019-12-08T12:36:39.374 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:39.376 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.386 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.387 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.398 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.410 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.411 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.432 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.434 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-08T12:36:39.435 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:39.437 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.447 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:39.770 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-08T12:36:39.771 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-08T12:36:39.773 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-08T12:36:39.804 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:39.806 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-08T12:36:39.827 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.139 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-08T12:36:40.144 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-08T12:36:40.163 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-08T12:36:40.471 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-08T12:36:40.472 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-08T12:36:40.528 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:40.530 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-08T12:36:40.571 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:40 +0000 (0:00:01.236) 0:01:25.598 ******* 2019-12-08T12:36:40.618 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.619 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.630 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.653 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.653 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.655 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.667 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.681 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.693 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.836 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-08T12:36:40.874 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.876 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.880 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.897 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.898 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.910 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.921 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.932 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.946 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.947 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.961 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.985 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.986 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:40.997 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.009 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.031 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-08T12:36:41.032 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.043 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.054 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.056 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.069 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.084 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.093 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.093 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.105 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.116 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.129 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.140 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.152 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.163 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.176 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.198 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.199 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.212 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.236 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.238 INFO:teuthology.task.ansible.out: skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.248 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.270 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.270 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.292 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.304 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.314 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.326 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.333 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.334 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.345 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.367 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.377 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.389 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.400 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.401 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.411 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.423 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.424 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-08T12:36:41.426 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.436 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.447 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.458 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.470 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.481 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.498 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.519 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.527 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.538 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.539 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.550 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.561 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.589 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.593 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.593 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.611 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.623 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.634 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.635 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.646 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.657 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.658 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.669 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.680 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.691 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.692 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.716 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.727 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.738 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.740 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.753 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.776 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.796 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-08T12:36:41.835 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:41 +0000 (0:00:01.268) 0:01:26.866 ******* 2019-12-08T12:36:41.868 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:41.902 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:41 +0000 (0:00:00.067) 0:01:26.933 ******* 2019-12-08T12:36:42.149 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:42.151 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-08T12:36:42.153 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.153 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.164 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.176 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.189 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.190 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.201 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.213 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.226 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.227 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.239 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.250 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.272 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-08T12:36:42.273 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-08T12:36:42.275 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:42.276 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.278 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.288 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.306 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.308 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.320 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.332 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.332 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.343 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.355 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.366 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.377 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.378 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.389 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.400 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.411 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.412 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.423 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.435 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.446 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.458 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.459 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.470 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.482 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.493 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.504 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.505 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.516 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.527 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.539 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.551 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.552 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.574 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-08T12:36:42.576 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-08T12:36:42.577 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.579 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.592 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.593 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.604 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.615 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.627 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.628 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.639 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.651 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.653 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.664 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.686 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.700 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.720 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.722 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.732 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.744 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.755 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.757 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.768 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.780 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.791 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.802 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.815 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.826 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.838 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.851 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.864 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.866 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.878 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.889 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.890 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi181.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-08T12:36:42.902 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-08T12:36:42.903 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.915 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.926 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.937 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.948 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.949 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.960 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.972 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.987 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:42.999 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:43.012 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:43.035 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:43.053 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:43.064 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:43.162 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:43 +0000 (0:00:01.259) 0:01:28.192 ******* 2019-12-08T12:36:43.189 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-08T12:36:43.190 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:43.228 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:43 +0000 (0:00:00.066) 0:01:28.259 ******* 2019-12-08T12:36:43.235 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-08T12:36:43.235 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-08T12:36:43.530 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-08T12:36:43.605 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-08T12:36:43.825 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-08T12:36:43.931 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-08T12:36:44.139 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-08T12:36:44.253 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-08T12:36:44.453 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-08T12:36:44.578 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-08T12:36:44.777 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-08T12:36:44.912 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-08T12:36:45.018 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-08T12:36:45.256 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-08T12:36:45.353 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-08T12:36:45.578 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-08T12:36:45.652 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-08T12:36:45.896 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-08T12:36:45.952 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-08T12:36:46.203 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-08T12:36:46.269 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-08T12:36:46.542 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-08T12:36:46.574 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-08T12:36:46.852 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-08T12:36:46.885 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-08T12:36:47.194 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-08T12:36:47.216 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-08T12:36:47.481 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-08T12:36:47.544 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-08T12:36:47.790 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-08T12:36:47.874 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-08T12:36:48.122 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-08T12:36:48.206 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-08T12:36:48.449 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-08T12:36:48.523 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-08T12:36:48.840 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-08T12:36:49.047 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-08T12:36:49.154 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-08T12:36:49.502 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-08T12:36:49.642 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-08T12:36:49.817 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-08T12:36:50.117 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-08T12:36:50.180 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-08T12:36:50.680 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-08T12:36:50.785 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-08T12:36:51.196 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-08T12:36:51.353 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-08T12:36:51.714 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-08T12:36:51.891 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-08T12:36:52.322 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-08T12:36:52.437 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-08T12:36:52.880 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-08T12:36:53.047 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-08T12:36:53.388 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-08T12:36:53.628 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-08T12:36:53.940 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-08T12:36:54.224 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-08T12:36:54.559 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-08T12:36:54.729 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-08T12:36:55.124 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-08T12:36:55.357 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-08T12:36:55.633 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-08T12:36:55.705 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-08T12:36:56.043 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-08T12:36:56.204 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-08T12:36:56.368 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-08T12:36:56.684 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-08T12:36:56.773 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-08T12:36:56.972 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-08T12:36:57.085 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-08T12:36:57.303 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-08T12:36:57.422 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-08T12:36:57.612 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-08T12:36:57.726 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-08T12:36:58.043 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-08T12:36:58.188 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-08T12:36:58.260 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:58 +0000 (0:00:15.029) 0:01:43.289 ******* 2019-12-08T12:36:58.369 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-08T12:36:58.696 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-08T12:36:59.039 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-08T12:36:59.369 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-08T12:36:59.371 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808618.28-96940095465233/source", "state": "file", "uid": 0} 2019-12-08T12:36:59.411 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:59 +0000 (0:00:01.151) 0:01:44.441 ******* 2019-12-08T12:36:59.438 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-08T12:36:59.439 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => { "msg": "Host smithi181.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-08T12:36:59.478 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:59 +0000 (0:00:00.067) 0:01:44.508 ******* 2019-12-08T12:36:59.515 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-08T12:36:59.552 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:59 +0000 (0:00:00.074) 0:01:44.582 ******* 2019-12-08T12:36:59.578 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-08T12:36:59.579 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:36:59.619 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:59 +0000 (0:00:00.064) 0:01:44.647 ******* 2019-12-08T12:36:59.622 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-08T12:36:59.623 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-08T12:36:59.683 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:36:59 +0000 (0:00:00.066) 0:01:44.713 ******* 2019-12-08T12:37:00.025 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-08T12:37:00.027 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.089789", "end": "2019-12-08 12:36:59.991597", "rc": 0, "start": "2019-12-08 12:36:59.901808", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-08T12:37:00.098 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.414) 0:01:45.128 ******* 2019-12-08T12:37:00.135 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-08T12:37:00.140 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.176 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.078) 0:01:45.207 ******* 2019-12-08T12:37:00.203 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-08T12:37:00.205 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.240 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.063) 0:01:45.270 ******* 2019-12-08T12:37:00.293 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-08T12:37:00.295 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.331 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.091) 0:01:45.362 ******* 2019-12-08T12:37:00.361 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.398 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.066) 0:01:45.428 ******* 2019-12-08T12:37:00.435 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-08T12:37:00.437 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.473 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.075) 0:01:45.504 ******* 2019-12-08T12:37:00.500 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-08T12:37:00.501 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.537 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.063) 0:01:45.568 ******* 2019-12-08T12:37:00.574 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi181.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:00.579 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:00.613 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.075) 0:01:45.643 ******* 2019-12-08T12:37:00.650 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-08T12:37:00.651 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.708 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.094) 0:01:45.738 ******* 2019-12-08T12:37:00.735 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-08T12:37:00.736 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.775 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.067) 0:01:45.805 ******* 2019-12-08T12:37:00.792 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-08T12:37:00.792 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808619.73-147364373000075/source", "state": "file", "uid": 0} 2019-12-08T12:37:00.836 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.061) 0:01:45.867 ******* 2019-12-08T12:37:00.847 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-08T12:37:00.849 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:00.888 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.051) 0:01:45.918 ******* 2019-12-08T12:37:00.894 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-08T12:37:00.896 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => { "msg": "Host smithi129.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-08T12:37:00.934 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.046) 0:01:45.965 ******* 2019-12-08T12:37:00.941 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-08T12:37:00.943 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:00.982 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:00 +0000 (0:00:00.047) 0:01:46.013 ******* 2019-12-08T12:37:00.989 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-08T12:37:00.991 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-08T12:37:01.038 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.055) 0:01:46.069 ******* 2019-12-08T12:37:01.048 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:01.084 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.046) 0:01:46.115 ******* 2019-12-08T12:37:01.090 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-08T12:37:01.091 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.140 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.055) 0:01:46.170 ******* 2019-12-08T12:37:01.153 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-08T12:37:01.154 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.197 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.057) 0:01:46.228 ******* 2019-12-08T12:37:01.224 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-08T12:37:01.225 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.263 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.066) 0:01:46.294 ******* 2019-12-08T12:37:01.300 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-08T12:37:01.301 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.358 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.095) 0:01:46.389 ******* 2019-12-08T12:37:01.396 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-08T12:37:01.397 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.434 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.075) 0:01:46.465 ******* 2019-12-08T12:37:01.462 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-08T12:37:01.463 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:01.501 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.066) 0:01:46.531 ******* 2019-12-08T12:37:01.528 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-08T12:37:01.528 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.567 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.066) 0:01:46.597 ******* 2019-12-08T12:37:01.572 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-08T12:37:01.574 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.089296", "end": "2019-12-08 12:37:01.529747", "rc": 0, "start": "2019-12-08 12:37:01.440451", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-08T12:37:01.613 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.045) 0:01:46.643 ******* 2019-12-08T12:37:01.620 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-08T12:37:01.621 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.659 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.046) 0:01:46.689 ******* 2019-12-08T12:37:01.664 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-08T12:37:01.665 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.707 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.047) 0:01:46.737 ******* 2019-12-08T12:37:01.719 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.754 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.047) 0:01:46.784 ******* 2019-12-08T12:37:01.759 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-08T12:37:01.760 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.800 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.046) 0:01:46.831 ******* 2019-12-08T12:37:01.806 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-08T12:37:01.808 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.845 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.045) 0:01:46.876 ******* 2019-12-08T12:37:01.851 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-08T12:37:01.852 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.891 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.045) 0:01:46.921 ******* 2019-12-08T12:37:01.897 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-08T12:37:01.898 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.936 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.044) 0:01:46.966 ******* 2019-12-08T12:37:01.940 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-08T12:37:01.943 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:01.982 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:01 +0000 (0:00:00.046) 0:01:47.012 ******* 2019-12-08T12:37:01.988 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-08T12:37:01.989 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.027 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.045) 0:01:47.058 ******* 2019-12-08T12:37:02.034 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-08T12:37:02.034 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.074 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.046) 0:01:47.104 ******* 2019-12-08T12:37:02.078 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-08T12:37:02.079 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:02.122 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.044) 0:01:47.149 ******* 2019-12-08T12:37:02.123 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-08T12:37:02.124 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.176 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.057) 0:01:47.207 ******* 2019-12-08T12:37:02.188 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-08T12:37:02.240 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.063) 0:01:47.270 ******* 2019-12-08T12:37:02.245 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-08T12:37:02.246 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:02.247 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:02.286 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.046) 0:01:47.317 ******* 2019-12-08T12:37:02.294 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-08T12:37:02.295 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.336 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.049) 0:01:47.366 ******* 2019-12-08T12:37:02.342 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-08T12:37:02.342 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.379 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.043) 0:01:47.410 ******* 2019-12-08T12:37:02.405 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-08T12:37:02.406 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.448 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.068) 0:01:47.478 ******* 2019-12-08T12:37:02.474 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-08T12:37:02.476 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:02.511 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.063) 0:01:47.542 ******* 2019-12-08T12:37:02.538 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-08T12:37:02.539 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.574 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.062) 0:01:47.604 ******* 2019-12-08T12:37:02.612 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-08T12:37:02.614 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:02.653 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.074) 0:01:47.679 ******* 2019-12-08T12:37:02.676 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-08T12:37:02.676 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:37:02.717 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.068) 0:01:47.748 ******* 2019-12-08T12:37:02.723 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-08T12:37:02.723 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.772 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.049) 0:01:47.798 ******* 2019-12-08T12:37:02.795 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.830 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.062) 0:01:47.861 ******* 2019-12-08T12:37:02.857 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-08T12:37:02.859 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.894 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.064) 0:01:47.925 ******* 2019-12-08T12:37:02.921 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-08T12:37:02.923 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:02.959 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:02 +0000 (0:00:00.064) 0:01:47.990 ******* 2019-12-08T12:37:02.986 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-08T12:37:02.987 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:03.021 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.062) 0:01:48.052 ******* 2019-12-08T12:37:03.048 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-08T12:37:03.050 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.093 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.071) 0:01:48.123 ******* 2019-12-08T12:37:03.130 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-08T12:37:03.132 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.168 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.075) 0:01:48.198 ******* 2019-12-08T12:37:03.195 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-08T12:37:03.196 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.237 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.068) 0:01:48.267 ******* 2019-12-08T12:37:03.276 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.315 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.078) 0:01:48.345 ******* 2019-12-08T12:37:03.345 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-08T12:37:03.347 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.382 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.067) 0:01:48.413 ******* 2019-12-08T12:37:03.423 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-08T12:37:03.425 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.467 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.085) 0:01:48.498 ******* 2019-12-08T12:37:03.494 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-08T12:37:03.495 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-08T12:37:03.538 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.070) 0:01:48.569 ******* 2019-12-08T12:37:03.564 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-08T12:37:03.604 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.066) 0:01:48.635 ******* 2019-12-08T12:37:03.644 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-08T12:37:03.646 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:03.689 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:03 +0000 (0:00:00.084) 0:01:48.719 ******* 2019-12-08T12:37:03.975 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-08T12:37:03.976 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-08T12:37:04.031 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:37:04.067 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:04 +0000 (0:00:00.378) 0:01:49.098 ******* 2019-12-08T12:37:04.846 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-08T12:37:04.847 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-08T12:37:04.890 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:04 +0000 (0:00:00.823) 0:01:49.921 ******* 2019-12-08T12:37:05.004 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-08T12:37:05.882 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-08T12:37:05.930 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:05 +0000 (0:00:01.039) 0:01:50.960 ******* 2019-12-08T12:37:05.967 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-08T12:37:05.968 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808624.93-206098201963776/source", "state": "file", "uid": 0} 2019-12-08T12:37:07.009 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808625.99-97518703757431/source", "state": "file", "uid": 0} 2019-12-08T12:37:07.060 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:07 +0000 (0:00:01.130) 0:01:52.091 ******* 2019-12-08T12:37:07.112 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808625.97-114862274613517/source", "state": "file", "uid": 0} 2019-12-08T12:37:07.698 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-08T12:37:07.699 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-08T12:37:08.025 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-12-08T12:37:08.059 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:08 +0000 (0:00:00.998) 0:01:53.090 ******* 2019-12-08T12:37:08.236 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* changed: [smithi129.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575808627.14-235938782855349/source", "state": "file", "uid": 0} 2019-12-08T12:37:08.277 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:08 +0000 (0:00:00.217) 0:01:53.308 ******* 2019-12-08T12:37:08.626 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-08T12:37:08.626 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-08T12:37:08.929 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-08T12:37:08.974 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:08 +0000 (0:00:00.697) 0:01:54.005 ******* 2019-12-08T12:37:22.092 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-12-08T12:37:22.094 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.790656", "end": "2019-12-08 12:37:22.062297", "rc": 0, "start": "2019-12-08 12:37:09.271641", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: ftp.osuosl.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: ftp.osuosl.org", " * extras: mirror.wdc1.us.leaseweb.net", " * updates: mirror.math.princeton.edu", "Metadata Cache Created"]} 2019-12-08T12:37:22.140 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:22 +0000 (0:00:13.165) 0:02:07.170 ******* 2019-12-08T12:37:22.515 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-08T12:37:22.517 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-08T12:37:22.829 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-08T12:37:22.900 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:22 +0000 (0:00:00.760) 0:02:07.931 ******* 2019-12-08T12:37:22.939 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:22.981 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:22 +0000 (0:00:00.081) 0:02:08.012 ******* 2019-12-08T12:37:23.019 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-08T12:37:23.020 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.059 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.076) 0:02:08.089 ******* 2019-12-08T12:37:23.086 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-08T12:37:23.087 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.128 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.064) 0:02:08.154 ******* 2019-12-08T12:37:23.160 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-08T12:37:23.162 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.199 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.075) 0:02:08.230 ******* 2019-12-08T12:37:23.226 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-08T12:37:23.227 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.289 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.089) 0:02:08.319 ******* 2019-12-08T12:37:23.291 INFO:teuthology.task.ansible.out: 2019-12-08T12:37:23.330 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-08T12:37:23.332 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.375 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.079) 0:02:08.399 ******* TASK [common : Update yum cache] *********************************************** 2019-12-08T12:37:23.376 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:14.962186", "end": "2019-12-08 12:37:23.324453", "rc": 0, "start": "2019-12-08 12:37:08.362267", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirrors.umflint.edu\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirrors.umflint.edu", " * extras: mirror.wdc1.us.leaseweb.net", " * updates: mirror.math.princeton.edu", "Metadata Cache Created"]} 2019-12-08T12:37:23.416 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.047) 0:02:08.446 ******* 2019-12-08T12:37:23.422 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-08T12:37:23.423 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.461 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.045) 0:02:08.491 ******* 2019-12-08T12:37:23.489 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-08T12:37:23.490 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:23.553 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:23 +0000 (0:00:00.091) 0:02:08.583 ******* 2019-12-08T12:37:23.791 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-08T12:37:24.097 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-08T12:37:24.147 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.594) 0:02:09.177 ******* 2019-12-08T12:37:24.176 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-08T12:37:24.178 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.231 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.083) 0:02:09.261 ******* 2019-12-08T12:37:24.260 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.316 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.083) 0:02:09.345 ******* 2019-12-08T12:37:24.344 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-08T12:37:24.345 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.384 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.069) 0:02:09.414 ******* 2019-12-08T12:37:24.410 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-08T12:37:24.416 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.453 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.064) 0:02:09.478 ******* 2019-12-08T12:37:24.475 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-08T12:37:24.476 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.513 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.065) 0:02:09.544 ******* 2019-12-08T12:37:24.540 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-08T12:37:24.541 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.579 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.065) 0:02:09.610 ******* 2019-12-08T12:37:24.620 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.655 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.076) 0:02:09.686 ******* 2019-12-08T12:37:24.681 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-08T12:37:24.683 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:24.720 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:24 +0000 (0:00:00.064) 0:02:09.750 ******* 2019-12-08T12:37:25.526 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.650984", "end": "2019-12-08 12:37:25.500442", "rc": 0, "start": "2019-12-08 12:37:23.849458", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:37:25.571 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:25 +0000 (0:00:00.850) 0:02:10.601 ******* 2019-12-08T12:37:26.439 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-08T12:37:26.440 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-08T12:37:26.495 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:26 +0000 (0:00:00.924) 0:02:11.526 ******* 2019-12-08T12:37:26.522 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-08T12:37:26.524 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:26.561 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:26 +0000 (0:00:00.065) 0:02:11.591 ******* 2019-12-08T12:37:26.571 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-08T12:37:26.571 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.504851", "end": "2019-12-08 12:37:26.500439", "rc": 0, "start": "2019-12-08 12:37:24.995588", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:37:26.615 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:26 +0000 (0:00:00.054) 0:02:11.646 ******* 2019-12-08T12:37:26.621 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-08T12:37:26.623 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:26.665 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:26 +0000 (0:00:00.046) 0:02:11.692 ******* 2019-12-08T12:37:26.689 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-08T12:37:26.691 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:26.732 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:26 +0000 (0:00:00.067) 0:02:11.760 ******* 2019-12-08T12:37:26.767 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-08T12:37:26.769 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:26.804 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:26 +0000 (0:00:00.074) 0:02:11.834 ******* 2019-12-08T12:37:27.141 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-08T12:37:27.142 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-08T12:37:27.191 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:27 +0000 (0:00:00.387) 0:02:12.222 ******* 2019-12-08T12:37:27.220 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-08T12:37:27.222 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:27.262 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:27 +0000 (0:00:00.070) 0:02:12.293 ******* 2019-12-08T12:37:27.300 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-08T12:37:27.324 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:27.352 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:27 +0000 (0:00:00.090) 0:02:12.383 ******* 2019-12-08T12:37:27.379 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-08T12:37:27.383 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:27.422 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:27 +0000 (0:00:00.069) 0:02:12.452 ******* 2019-12-08T12:37:27.473 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-08T12:37:27.476 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:27.519 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:27 +0000 (0:00:00.097) 0:02:12.549 ******* 2019-12-08T12:37:27.718 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-08T12:37:27.719 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-08T12:37:27.764 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:27 +0000 (0:00:00.245) 0:02:12.795 ******* 2019-12-08T12:37:28.397 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-08T12:37:28.448 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:28 +0000 (0:00:00.683) 0:02:13.478 ******* 2019-12-08T12:37:31.520 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-08T12:37:31.522 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-08T12:37:31.559 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:31 +0000 (0:00:03.110) 0:02:16.589 ******* 2019-12-08T12:37:32.375 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-08T12:37:32.414 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:32 +0000 (0:00:00.855) 0:02:17.445 ******* 2019-12-08T12:37:32.501 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-08T12:37:32.501 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-08T12:37:33.344 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-08T12:37:33.366 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-08T12:37:34.261 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-08T12:37:34.306 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:34 +0000 (0:00:01.892) 0:02:19.337 ******* 2019-12-08T12:37:34.330 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-08T12:37:34.663 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-08T12:37:34.665 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-08T12:37:34.700 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:34 +0000 (0:00:00.393) 0:02:19.731 ******* 2019-12-08T12:37:35.178 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-08T12:37:35.178 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-08T12:37:35.226 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:35 +0000 (0:00:00.525) 0:02:20.257 ******* 2019-12-08T12:37:35.549 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-08T12:37:35.551 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-08T12:37:35.592 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:35 +0000 (0:00:00.365) 0:02:20.623 ******* 2019-12-08T12:37:35.599 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-08T12:37:35.600 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-08T12:37:36.442 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-08T12:37:36.475 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-08T12:37:37.320 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-08T12:37:37.344 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-08T12:37:37.429 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:37 +0000 (0:00:01.836) 0:02:22.459 ******* 2019-12-08T12:37:37.469 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-08T12:37:37.471 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:37.539 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:37 +0000 (0:00:00.110) 0:02:22.570 ******* 2019-12-08T12:37:37.578 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-08T12:37:37.580 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-08T12:37:37.617 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:37 +0000 (0:00:00.077) 0:02:22.647 ******* 2019-12-08T12:37:37.947 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-08T12:37:37.949 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-08T12:37:37.988 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:37 +0000 (0:00:00.371) 0:02:23.019 ******* 2019-12-08T12:37:38.032 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi129.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.039 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.061 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.073 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.111 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:38 +0000 (0:00:00.123) 0:02:23.142 ******* 2019-12-08T12:37:38.172 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-08T12:37:38.173 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-08T12:37:38.217 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:38 +0000 (0:00:00.104) 0:02:23.247 ******* 2019-12-08T12:37:38.243 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-08T12:37:38.244 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.283 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:38 +0000 (0:00:00.066) 0:02:23.313 ******* 2019-12-08T12:37:38.320 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-08T12:37:38.322 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-08T12:37:38.362 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:38 +0000 (0:00:00.078) 0:02:23.392 ******* 2019-12-08T12:37:38.680 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi181.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-08T12:37:38.716 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:38 +0000 (0:00:00.354) 0:02:23.747 ******* 2019-12-08T12:37:38.774 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-08T12:37:38.775 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.905 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.916 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.937 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-08T12:37:38.979 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:38 +0000 (0:00:00.261) 0:02:24.009 ******* 2019-12-08T12:37:41.560 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-08T12:37:41.561 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-08T12:37:41.599 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:41 +0000 (0:00:02.620) 0:02:26.630 ******* 2019-12-08T12:37:41.636 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-08T12:37:41.637 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:41.679 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:41 +0000 (0:00:00.080) 0:02:26.710 ******* 2019-12-08T12:37:42.315 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-08T12:37:42.316 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-08T12:37:42.360 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:42 +0000 (0:00:00.681) 0:02:27.391 ******* 2019-12-08T12:37:42.390 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-08T12:37:42.391 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:42.442 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:42 +0000 (0:00:00.081) 0:02:27.472 ******* 2019-12-08T12:37:42.728 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-08T12:37:42.730 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-08T12:37:42.890 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:42 +0000 (0:00:00.448) 0:02:27.921 ******* 2019-12-08T12:37:43.494 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-08T12:37:43.547 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:43 +0000 (0:00:00.656) 0:02:28.577 ******* 2019-12-08T12:37:43.806 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-08T12:37:43.807 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-08T12:37:43.867 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:43 +0000 (0:00:00.319) 0:02:28.897 ******* 2019-12-08T12:37:43.913 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-08T12:37:43.915 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:43.951 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:43 +0000 (0:00:00.084) 0:02:28.982 ******* 2019-12-08T12:37:44.271 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-08T12:37:44.273 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:37:44.311 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:44 +0000 (0:00:00.359) 0:02:29.341 ******* 2019-12-08T12:37:44.488 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-08T12:37:44.549 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:44 +0000 (0:00:00.238) 0:02:29.580 ******* 2019-12-08T12:37:44.586 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-08T12:37:44.588 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:44.627 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:44 +0000 (0:00:00.077) 0:02:29.657 ******* 2019-12-08T12:37:44.664 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-08T12:37:44.665 INFO:teuthology.task.ansible.out:fatal: [smithi129.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-08T12:37:44.702 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:44 +0000 (0:00:00.075) 0:02:29.733 ******* 2019-12-08T12:37:44.742 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-08T12:37:44.743 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:44.784 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:44 +0000 (0:00:00.081) 0:02:29.814 ******* 2019-12-08T12:37:44.947 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:37:44.985 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:44 +0000 (0:00:00.201) 0:02:30.016 ******* 2019-12-08T12:37:45.344 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-08T12:37:45.346 INFO:teuthology.task.ansible.out:fatal: [smithi181.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-08T12:37:45.385 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:45 +0000 (0:00:00.400) 0:02:30.416 ******* 2019-12-08T12:37:45.422 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-08T12:37:45.424 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:37:45.462 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:45 +0000 (0:00:00.076) 0:02:30.492 ******* 2019-12-08T12:37:45.675 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-08T12:37:45.675 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-08T12:37:45.718 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:45 +0000 (0:00:00.256) 0:02:30.749 ******* 2019-12-08T12:37:46.403 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-08T12:37:46.440 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:46 +0000 (0:00:00.721) 0:02:31.471 ******* 2019-12-08T12:37:46.498 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-08T12:37:46.500 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-12-08 12:34:36 UTC", "ActiveEnterTimestampMonotonic": "19437129", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "local-fs.target systemd-journald.socket var-run.mount time-sync.target network.target system.slice nss-lookup.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-12-08 12:34:36 UTC", "AssertTimestampMonotonic": "19436226", "Before": "getty@tty1.service multi-user.target xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-12-08 12:34:36 UTC", "ConditionTimestampMonotonic": "19436225", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8957", "ExecMainStartTimestamp": "Sun 2019-12-08 12:34:36 UTC", "ExecMainStartTimestampMonotonic": "19437064", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-08T12:37:46.501 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Sun 2019-12-08 12:34:36 UTC] ; stop_time=[n/a] ; pid=8957 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2019-12-08 12:34:36 UTC", "InactiveExitTimestampMonotonic": "19437129", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8957", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target network-online.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-08T12:37:46.502 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-12-08 12:34:36 UTC", "WatchdogTimestampMonotonic": "19437107", "WatchdogUSec": "0"}} 2019-12-08T12:37:46.541 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:46 +0000 (0:00:00.100) 0:02:31.571 ******* 2019-12-08T12:37:46.852 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-12-08 12:34:40 UTC", "ActiveEnterTimestampMonotonic": "19671737", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target local-fs.target network.target time-sync.target system.slice var-run.mount systemd-journald.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-12-08 12:34:40 UTC", "AssertTimestampMonotonic": "19670906", "Before": "shutdown.target getty@tty1.service xdm.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-12-08 12:34:40 UTC", "ConditionTimestampMonotonic": "19670903", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8942", "ExecMainStartTimestamp": "Sun 2019-12-08 12:34:40 UTC", "ExecMainStartTimestampMonotonic": "19671664", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-08T12:37:46.854 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Sun 2019-12-08 12:34:40 UTC] ; stop_time=[n/a] ; pid=8942 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2019-12-08 12:34:40 UTC", "InactiveExitTimestampMonotonic": "19671737", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8942", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target network-online.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-08T12:37:46.855 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-12-08 12:34:40 UTC", "WatchdogTimestampMonotonic": "19671710", "WatchdogUSec": "0"}} TASK [common : Get SELinux status] ********************************************* changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.003857", "end": "2019-12-08 12:37:46.830420", "rc": 0, "start": "2019-12-08 12:37:46.826563", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-08T12:37:46.898 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:46 +0000 (0:00:00.357) 0:02:31.929 ******* 2019-12-08T12:37:46.945 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:46 +0000 (0:00:00.046) 0:02:31.976 ******* 2019-12-08T12:37:47.260 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004798", "end": "2019-12-08 12:37:47.231334", "rc": 0, "start": "2019-12-08 12:37:47.226536", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-08T12:37:47.301 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:47 +0000 (0:00:00.356) 0:02:32.332 ******* 2019-12-08T12:37:47.392 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-08T12:37:47.394 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-08T12:37:47.441 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:47 +0000 (0:00:00.139) 0:02:32.471 ******* 2019-12-08T12:37:47.826 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-08T12:37:47.865 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:37:47 +0000 (0:00:00.424) 0:02:32.896 ******* 2019-12-08T12:38:14.903 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-08T12:38:14.903 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: ftp.osuosl.org\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-08T12:38:14.907 INFO:teuthology.task.ansible.out: 15 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-08T12:38:14.950 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:14 +0000 (0:00:27.084) 0:02:59.980 ******* 2019-12-08T12:38:15.319 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.umflint.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-08T12:38:15.320 INFO:teuthology.task.ansible.out: 13 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-08T12:38:15.359 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:15 +0000 (0:00:00.409) 0:03:00.390 ******* 2019-12-08T12:38:15.622 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-08T12:38:15.665 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:15 +0000 (0:00:00.305) 0:03:00.695 ******* 2019-12-08T12:38:15.874 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-08T12:38:15.926 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:15 +0000 (0:00:00.261) 0:03:00.957 ******* 2019-12-08T12:38:23.750 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-08T12:38:23.751 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.782126", "end": "2019-12-08 12:38:23.718191", "failed_when_result": false, "rc": 0, "start": "2019-12-08 12:38:15.936065", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-08T12:38:23.788 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:23 +0000 (0:00:07.861) 0:03:08.819 ******* 2019-12-08T12:38:24.055 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.818604", "end": "2019-12-08 12:38:24.022331", "failed_when_result": false, "rc": 0, "start": "2019-12-08 12:38:16.203727", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-08T12:38:24.097 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:24 +0000 (0:00:00.307) 0:03:09.126 ******* 2019-12-08T12:38:24.771 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-08T12:38:24.810 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:24 +0000 (0:00:00.714) 0:03:09.841 ******* 2019-12-08T12:38:25.066 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-08T12:38:25.121 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:25 +0000 (0:00:00.311) 0:03:10.152 ******* 2019-12-08T12:38:25.169 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-08T12:38:25.170 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.023583", "end": "2019-12-08 12:38:25.142924", "rc": 0, "start": "2019-12-08 12:38:25.119341", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-08T12:38:25.217 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:25 +0000 (0:00:00.095) 0:03:10.248 ******* 2019-12-08T12:38:25.435 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.031885", "end": "2019-12-08 12:38:25.407737", "rc": 0, "start": "2019-12-08 12:38:25.375852", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-08T12:38:25.481 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:25 +0000 (0:00:00.262) 0:03:10.511 ******* 2019-12-08T12:38:25.532 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-08T12:38:25.532 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.022389", "end": "2019-12-08 12:38:25.511322", "rc": 0, "start": "2019-12-08 12:38:25.488933", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:38:25.574 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:25 +0000 (0:00:00.092) 0:03:10.604 ******* 2019-12-08T12:38:25.809 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.011057", "end": "2019-12-08 12:38:25.787457", "rc": 0, "start": "2019-12-08 12:38:25.776400", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:38:25.857 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:25 +0000 (0:00:00.283) 0:03:10.888 ******* 2019-12-08T12:38:33.780 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-08T12:38:33.781 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.883720", "end": "2019-12-08 12:38:33.747122", "rc": 0, "start": "2019-12-08 12:38:25.863402", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:38:33.819 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:33 +0000 (0:00:07.961) 0:03:18.849 ******* 2019-12-08T12:38:34.146 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.969452", "end": "2019-12-08 12:38:34.115427", "rc": 0, "start": "2019-12-08 12:38:26.145975", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:38:34.183 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.364) 0:03:19.214 ******* 2019-12-08T12:38:34.188 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-08T12:38:34.189 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-08T12:38:34.248 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.065) 0:03:19.279 ******* 2019-12-08T12:38:34.287 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-08T12:38:34.287 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-08T12:38:34.326 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.077) 0:03:19.357 ******* 2019-12-08T12:38:34.363 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-08T12:38:34.364 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-08T12:38:34.400 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.074) 0:03:19.431 ******* 2019-12-08T12:38:34.440 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-08T12:38:34.441 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-08T12:38:34.484 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.083) 0:03:19.514 ******* 2019-12-08T12:38:34.521 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-08T12:38:34.522 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-08T12:38:34.559 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.074) 0:03:19.589 ******* 2019-12-08T12:38:34.565 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-08T12:38:34.566 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-08T12:38:34.604 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.045) 0:03:19.635 ******* 2019-12-08T12:38:34.631 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-08T12:38:34.632 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-08T12:38:34.671 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.066) 0:03:19.702 ******* 2019-12-08T12:38:34.709 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-08T12:38:34.710 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-08T12:38:34.746 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.075) 0:03:19.777 ******* 2019-12-08T12:38:34.790 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-08T12:38:34.791 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-08T12:38:34.828 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.082) 0:03:19.859 ******* 2019-12-08T12:38:34.876 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-08T12:38:34.878 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} TASK [testnode : Including version specific variables.] ************************ ok: [smithi181.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-08T12:38:34.915 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.086) 0:03:19.945 ******* 2019-12-08T12:38:34.948 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:34 +0000 (0:00:00.033) 0:03:19.979 ******* 2019-12-08T12:38:35.223 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-08T12:38:35.225 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-08T12:38:35.262 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.314) 0:03:20.293 ******* 2019-12-08T12:38:35.268 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-08T12:38:35.270 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-08T12:38:35.306 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.043) 0:03:20.337 ******* 2019-12-08T12:38:35.310 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-08T12:38:35.312 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:35.350 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.043) 0:03:20.381 ******* 2019-12-08T12:38:35.387 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-08T12:38:35.389 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:35.429 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.079) 0:03:20.460 ******* 2019-12-08T12:38:35.623 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-08T12:38:35.625 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-08T12:38:35.661 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.231) 0:03:20.691 ******* 2019-12-08T12:38:35.687 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-08T12:38:35.689 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:35.724 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.062) 0:03:20.754 ******* 2019-12-08T12:38:35.750 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-08T12:38:35.752 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:35.788 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.064) 0:03:20.819 ******* 2019-12-08T12:38:35.793 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-08T12:38:35.794 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-08T12:38:35.832 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:35 +0000 (0:00:00.043) 0:03:20.862 ******* 2019-12-08T12:38:36.128 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-08T12:38:36.165 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:36 +0000 (0:00:00.333) 0:03:21.196 ******* 2019-12-08T12:38:36.835 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-08T12:38:36.835 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-08T12:38:36.872 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:36 +0000 (0:00:00.706) 0:03:21.903 ******* 2019-12-08T12:38:36.910 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-08T12:38:36.912 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi129"}, "changed": false} 2019-12-08T12:38:36.948 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:36 +0000 (0:00:00.076) 0:03:21.979 ******* 2019-12-08T12:38:37.142 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-08T12:38:37.143 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-08T12:38:37.187 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:37 +0000 (0:00:00.238) 0:03:22.218 ******* 2019-12-08T12:38:37.226 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-08T12:38:37.227 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi181"}, "changed": false} 2019-12-08T12:38:37.265 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:37 +0000 (0:00:00.077) 0:03:22.296 ******* 2019-12-08T12:38:37.896 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-08T12:38:37.898 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi181.front.sepia.ceph.com", "ansible_hostname": "smithi181", "ansible_nodename": "smithi181"}, "changed": true, "name": "smithi181"} 2019-12-08T12:38:37.937 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:37 +0000 (0:00:00.671) 0:03:22.967 ******* 2019-12-08T12:38:37.943 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi129.front.sepia.ceph.com", "ansible_hostname": "smithi129", "ansible_nodename": "smithi129"}, "changed": true, "name": "smithi129"} 2019-12-08T12:38:37.982 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:37 +0000 (0:00:00.045) 0:03:23.013 ******* 2019-12-08T12:38:38.821 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-08T12:38:38.822 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-08T12:38:38.867 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:38 +0000 (0:00:00.884) 0:03:23.897 ******* 2019-12-08T12:38:38.873 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-08T12:38:38.914 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:38 +0000 (0:00:00.047) 0:03:23.944 ******* 2019-12-08T12:38:39.701 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-08T12:38:39.702 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-08T12:38:39.743 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:39 +0000 (0:00:00.828) 0:03:24.773 ******* 2019-12-08T12:38:39.841 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-08T12:38:39.883 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:39 +0000 (0:00:00.140) 0:03:24.914 ******* 2019-12-08T12:38:40.604 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-08T12:38:40.605 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-08T12:38:40.643 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:40 +0000 (0:00:00.759) 0:03:25.673 ******* 2019-12-08T12:38:40.776 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-08T12:38:40.820 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:40 +0000 (0:00:00.177) 0:03:25.851 ******* 2019-12-08T12:38:41.563 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-08T12:38:41.564 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-08T12:38:41.601 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:41 +0000 (0:00:00.780) 0:03:26.631 ******* 2019-12-08T12:38:41.769 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-08T12:38:41.808 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:41 +0000 (0:00:00.206) 0:03:26.838 ******* 2019-12-08T12:38:41.919 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-08T12:38:41.920 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-08T12:38:41.961 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:41 +0000 (0:00:00.152) 0:03:26.991 ******* 2019-12-08T12:38:42.133 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-08T12:38:42.173 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:42 +0000 (0:00:00.212) 0:03:27.204 ******* 2019-12-08T12:38:42.263 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-08T12:38:42.264 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-08T12:38:42.305 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:42 +0000 (0:00:00.131) 0:03:27.335 ******* 2019-12-08T12:38:42.498 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-08T12:38:42.535 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:42 +0000 (0:00:00.230) 0:03:27.566 ******* 2019-12-08T12:38:42.670 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-08T12:38:42.671 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-08T12:38:42.717 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:42 +0000 (0:00:00.181) 0:03:27.748 ******* 2019-12-08T12:38:42.900 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-08T12:38:42.940 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:42 +0000 (0:00:00.223) 0:03:27.971 ******* 2019-12-08T12:38:43.591 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-08T12:38:43.592 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-08T12:38:43.638 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:43 +0000 (0:00:00.697) 0:03:28.669 ******* 2019-12-08T12:38:43.857 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-08T12:38:43.903 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:43 +0000 (0:00:00.264) 0:03:28.933 ******* 2019-12-08T12:38:44.529 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-08T12:38:44.531 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-08T12:38:44.566 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:44 +0000 (0:00:00.663) 0:03:29.597 ******* 2019-12-08T12:38:44.825 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-08T12:38:44.867 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:44 +0000 (0:00:00.300) 0:03:29.897 ******* 2019-12-08T12:38:46.972 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-08T12:38:46.974 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.073192", "end": "2019-12-08 12:38:46.939543", "rc": 0, "start": "2019-12-08 12:38:44.866351", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:38:47.010 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:47 +0000 (0:00:02.143) 0:03:32.041 ******* 2019-12-08T12:38:47.037 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-08T12:38:47.077 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:47 +0000 (0:00:00.066) 0:03:32.107 ******* 2019-12-08T12:38:47.322 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-08T12:38:47.323 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.147417", "end": "2019-12-08 12:38:47.294944", "rc": 0, "start": "2019-12-08 12:38:45.147527", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:38:47.360 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:47 +0000 (0:00:00.282) 0:03:32.390 ******* 2019-12-08T12:38:47.386 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-08T12:38:47.432 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:47 +0000 (0:00:00.071) 0:03:32.462 ******* 2019-12-08T12:38:48.015 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-08T12:38:48.016 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-08T12:38:48.362 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-08T12:38:48.819 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-08T12:38:48.861 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:48 +0000 (0:00:01.429) 0:03:33.892 ******* 2019-12-08T12:38:48.889 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-08T12:38:48.890 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:48.927 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:48 +0000 (0:00:00.065) 0:03:33.957 ******* 2019-12-08T12:38:48.966 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-08T12:38:48.968 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-08T12:38:48.978 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-08T12:38:49.024 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.096) 0:03:34.054 ******* 2019-12-08T12:38:49.050 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-08T12:38:49.052 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:49.088 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.064) 0:03:34.119 ******* 2019-12-08T12:38:49.125 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:38:49.127 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-08T12:38:49.162 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.074) 0:03:34.193 ******* 2019-12-08T12:38:49.242 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-08T12:38:49.243 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-08T12:38:49.283 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.120) 0:03:34.314 ******* 2019-12-08T12:38:49.310 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:49.348 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.064) 0:03:34.379 ******* 2019-12-08T12:38:49.386 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-08T12:38:49.387 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-08T12:38:49.401 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-08T12:38:49.445 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.095) 0:03:34.475 ******* 2019-12-08T12:38:49.471 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-08T12:38:49.473 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:49.509 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.064) 0:03:34.540 ******* 2019-12-08T12:38:49.546 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:38:49.548 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-08T12:38:49.584 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.075) 0:03:34.615 ******* 2019-12-08T12:38:49.633 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-08T12:38:49.634 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-08T12:38:49.674 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:49 +0000 (0:00:00.089) 0:03:34.704 ******* 2019-12-08T12:38:50.087 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-08T12:38:50.132 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:50 +0000 (0:00:00.458) 0:03:35.163 ******* 2019-12-08T12:38:50.326 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-08T12:38:50.328 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.326852", "end": "2019-12-08 12:38:50.285499", "rc": 0, "start": "2019-12-08 12:38:49.958647", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.umflint.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirrors.umflint.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.wdc1.us.leaseweb.net", " * updates: mirror.math.princeton.edu", "No unfinished transactions left."]} 2019-12-08T12:38:50.366 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:50 +0000 (0:00:00.233) 0:03:35.396 ******* 2019-12-08T12:38:50.705 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-08T12:38:50.706 INFO:teuthology.task.ansible.out:fatal: [smithi181.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.036211", "end": "2019-12-08 12:38:50.682338", "msg": "non-zero return code", "rc": 1, "start": "2019-12-08 12:38:50.646127", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-08T12:38:50.754 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:50 +0000 (0:00:00.387) 0:03:35.784 ******* 2019-12-08T12:38:50.760 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-08T12:38:50.761 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.296621", "end": "2019-12-08 12:38:50.706389", "rc": 0, "start": "2019-12-08 12:38:50.409768", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: ftp.osuosl.org\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: ftp.osuosl.org", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.wdc1.us.leaseweb.net", " * updates: mirror.math.princeton.edu", "No unfinished transactions left."]} 2019-12-08T12:38:50.805 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:50 +0000 (0:00:00.050) 0:03:35.835 ******* 2019-12-08T12:38:50.810 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-08T12:38:50.811 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:50.850 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:50 +0000 (0:00:00.045) 0:03:35.880 ******* 2019-12-08T12:38:51.189 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* fatal: [smithi129.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.052981", "end": "2019-12-08 12:38:51.163124", "msg": "non-zero return code", "rc": 1, "start": "2019-12-08 12:38:51.110143", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-08T12:38:51.228 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:51 +0000 (0:00:00.377) 0:03:36.258 ******* 2019-12-08T12:38:51.267 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:38:51.304 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:51 +0000 (0:00:00.076) 0:03:36.335 ******* 2019-12-08T12:38:59.611 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-08T12:38:59.612 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-08T12:38:59.613 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-08T12:38:59.615 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-08T12:38:59.654 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:59 +0000 (0:00:08.350) 0:03:44.685 ******* 2019-12-08T12:38:59.745 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-08T12:38:59.747 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-08T12:38:59.748 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-08T12:38:59.793 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:38:59 +0000 (0:00:00.138) 0:03:44.823 ******* 2019-12-08T12:39:02.978 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-08T12:39:02.979 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-08T12:39:03.027 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:03 +0000 (0:00:03.234) 0:03:48.058 ******* 2019-12-08T12:39:03.139 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-08T12:39:03.180 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:03 +0000 (0:00:00.152) 0:03:48.211 ******* 2019-12-08T12:39:21.114 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-08T12:39:21.115 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-08T12:39:21.117 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-08T12:39:21.118 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: ftp.osuosl.org\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies 2019-12-08T12:39:21.120 INFO:teuthology.task.ansible.out: Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 19 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-08T12:39:21.121 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-08T12:39:21.155 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:21 +0000 (0:00:17.975) 0:04:06.186 ******* 2019-12-08T12:39:21.917 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-08T12:39:21.919 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-08T12:39:21.956 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:21 +0000 (0:00:00.800) 0:04:06.986 ******* 2019-12-08T12:39:21.993 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-08T12:39:21.995 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-08T12:39:21.996 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-08T12:39:21.998 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.umflint.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.wdc1.us.leaseweb.net\n * updates: mirror.math.princeton.edu\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDepende 2019-12-08T12:39:21.999 INFO:teuthology.task.ansible.out:ncies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 15 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-08T12:39:22.001 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-08T12:39:22.034 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:22 +0000 (0:00:00.078) 0:04:07.065 ******* 2019-12-08T12:39:22.499 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-08T12:39:22.500 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-08T12:39:22.539 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:22 +0000 (0:00:00.504) 0:04:07.569 ******* 2019-12-08T12:39:22.575 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-08T12:39:22.577 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:22.615 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:22 +0000 (0:00:00.076) 0:04:07.646 ******* 2019-12-08T12:39:22.838 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-08T12:39:22.877 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:22 +0000 (0:00:00.261) 0:04:07.908 ******* 2019-12-08T12:39:23.061 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-08T12:39:23.063 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.service system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ipset.service iptables.service ebtables.service ip6tables.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-08T12:39:23.065 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-08T12:39:23.066 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-08T12:39:23.102 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:23 +0000 (0:00:00.224) 0:04:08.132 ******* 2019-12-08T12:39:23.415 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-08T12:39:23.453 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:23 +0000 (0:00:00.351) 0:04:08.484 ******* 2019-12-08T12:39:23.462 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-08T12:39:23.463 INFO:teuthology.task.ansible.out:fatal: [smithi129.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-08T12:39:23.509 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:23 +0000 (0:00:00.055) 0:04:08.540 ******* 2019-12-08T12:39:23.515 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-08T12:39:23.516 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:23.553 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:23 +0000 (0:00:00.043) 0:04:08.584 ******* 2019-12-08T12:39:23.981 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-08T12:39:23.983 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service basic.target dbus.service system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ebtables.service ip6tables.service shutdown.target ipset.service iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-08T12:39:23.985 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-08T12:39:23.986 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-08T12:39:24.019 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.466) 0:04:09.050 ******* 2019-12-08T12:39:24.280 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-08T12:39:24.281 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-08T12:39:24.329 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.309) 0:04:09.360 ******* 2019-12-08T12:39:24.357 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-08T12:39:24.358 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.396 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.066) 0:04:09.426 ******* 2019-12-08T12:39:24.413 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-08T12:39:24.413 INFO:teuthology.task.ansible.out:fatal: [smithi181.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-08T12:39:24.415 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-08T12:39:24.455 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.059) 0:04:09.485 ******* 2019-12-08T12:39:24.502 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.046) 0:04:09.532 ******* 2019-12-08T12:39:24.507 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-08T12:39:24.507 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.549 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.047) 0:04:09.580 ******* 2019-12-08T12:39:24.577 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-08T12:39:24.579 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.618 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.068) 0:04:09.648 ******* 2019-12-08T12:39:24.656 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-08T12:39:24.657 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.696 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.078) 0:04:09.726 ******* 2019-12-08T12:39:24.723 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-08T12:39:24.724 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.760 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.064) 0:04:09.791 ******* 2019-12-08T12:39:24.787 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-08T12:39:24.824 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.064) 0:04:09.855 ******* 2019-12-08T12:39:24.851 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-08T12:39:24.853 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.892 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.067) 0:04:09.922 ******* 2019-12-08T12:39:24.908 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-08T12:39:24.910 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-08T12:39:24.946 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.054) 0:04:09.977 ******* 2019-12-08T12:39:24.953 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:24.990 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:24 +0000 (0:00:00.044) 0:04:10.021 ******* 2019-12-08T12:39:24.997 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-08T12:39:24.999 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.037 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.046) 0:04:10.068 ******* 2019-12-08T12:39:25.044 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-08T12:39:25.044 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.084 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.046) 0:04:10.114 ******* 2019-12-08T12:39:25.090 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-08T12:39:25.130 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.045) 0:04:10.160 ******* 2019-12-08T12:39:25.140 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.177 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.047) 0:04:10.207 ******* 2019-12-08T12:39:25.183 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-08T12:39:25.183 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.223 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.046) 0:04:10.254 ******* 2019-12-08T12:39:25.228 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-08T12:39:25.230 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.267 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.043) 0:04:10.298 ******* 2019-12-08T12:39:25.273 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-08T12:39:25.275 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.313 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.045) 0:04:10.343 ******* 2019-12-08T12:39:25.318 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-08T12:39:25.319 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.360 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.046) 0:04:10.390 ******* 2019-12-08T12:39:25.365 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-08T12:39:25.366 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.403 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.043) 0:04:10.434 ******* 2019-12-08T12:39:25.408 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-08T12:39:25.410 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.449 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.045) 0:04:10.479 ******* 2019-12-08T12:39:25.455 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-08T12:39:25.464 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.503 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.052) 0:04:10.532 ******* 2019-12-08T12:39:25.507 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-08T12:39:25.507 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.548 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.046) 0:04:10.578 ******* 2019-12-08T12:39:25.553 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-08T12:39:25.596 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.048) 0:04:10.626 ******* 2019-12-08T12:39:25.602 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-08T12:39:25.604 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.643 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.047) 0:04:10.673 ******* 2019-12-08T12:39:25.650 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-08T12:39:25.651 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.690 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.047) 0:04:10.721 ******* 2019-12-08T12:39:25.696 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-08T12:39:25.699 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.740 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.049) 0:04:10.770 ******* 2019-12-08T12:39:25.747 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-08T12:39:25.748 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.788 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.047) 0:04:10.818 ******* 2019-12-08T12:39:25.794 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-08T12:39:25.796 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.836 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.048) 0:04:10.866 ******* 2019-12-08T12:39:25.842 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-08T12:39:25.844 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.885 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.048) 0:04:10.915 ******* 2019-12-08T12:39:25.891 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-08T12:39:25.892 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.937 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.049) 0:04:10.964 ******* 2019-12-08T12:39:25.942 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-08T12:39:25.943 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:25.993 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:25 +0000 (0:00:00.059) 0:04:11.024 ******* 2019-12-08T12:39:25.999 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-08T12:39:26.001 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.040 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.046) 0:04:11.070 ******* 2019-12-08T12:39:26.046 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-08T12:39:26.048 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.089 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.048) 0:04:11.120 ******* 2019-12-08T12:39:26.095 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-08T12:39:26.095 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.140 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.050) 0:04:11.171 ******* 2019-12-08T12:39:26.150 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.187 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.046) 0:04:11.217 ******* 2019-12-08T12:39:26.195 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-08T12:39:26.196 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.233 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.046) 0:04:11.263 ******* 2019-12-08T12:39:26.239 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-08T12:39:26.239 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.280 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.047) 0:04:11.311 ******* 2019-12-08T12:39:26.286 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-08T12:39:26.287 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.326 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.045) 0:04:11.357 ******* 2019-12-08T12:39:26.335 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-08T12:39:26.336 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.373 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.046) 0:04:11.404 ******* 2019-12-08T12:39:26.379 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-08T12:39:26.379 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.418 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.449 ******* 2019-12-08T12:39:26.426 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-08T12:39:26.427 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.467 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.493 ******* 2019-12-08T12:39:26.476 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.515 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.052) 0:04:11.546 ******* 2019-12-08T12:39:26.521 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-08T12:39:26.522 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.560 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.045) 0:04:11.591 ******* 2019-12-08T12:39:26.566 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-08T12:39:26.567 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.605 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.636 ******* 2019-12-08T12:39:26.611 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-08T12:39:26.613 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.653 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.680 ******* 2019-12-08T12:39:26.655 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-08T12:39:26.658 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.695 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.725 ******* 2019-12-08T12:39:26.701 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-08T12:39:26.702 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.739 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.770 ******* 2019-12-08T12:39:26.744 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-08T12:39:26.746 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.783 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.044) 0:04:11.814 ******* 2019-12-08T12:39:26.788 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-08T12:39:26.790 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.826 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.042) 0:04:11.857 ******* 2019-12-08T12:39:26.831 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-08T12:39:26.832 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.872 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.045) 0:04:11.903 ******* 2019-12-08T12:39:26.886 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-08T12:39:26.888 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.921 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.049) 0:04:11.952 ******* 2019-12-08T12:39:26.926 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-08T12:39:26.928 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:26.964 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:26 +0000 (0:00:00.042) 0:04:11.994 ******* 2019-12-08T12:39:26.969 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-08T12:39:26.971 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.014 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.048) 0:04:12.043 ******* 2019-12-08T12:39:27.018 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-08T12:39:27.019 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.059 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.046) 0:04:12.090 ******* 2019-12-08T12:39:27.064 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-08T12:39:27.066 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.102 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.043) 0:04:12.133 ******* 2019-12-08T12:39:27.109 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-08T12:39:27.110 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.148 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.046) 0:04:12.179 ******* 2019-12-08T12:39:27.154 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-08T12:39:27.155 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.195 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.045) 0:04:12.225 ******* 2019-12-08T12:39:27.200 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.244 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.045) 0:04:12.270 ******* 2019-12-08T12:39:27.246 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-08T12:39:27.248 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.287 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.046) 0:04:12.317 ******* 2019-12-08T12:39:27.312 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-08T12:39:27.313 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.352 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.065) 0:04:12.382 ******* 2019-12-08T12:39:27.379 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-08T12:39:27.380 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.416 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.064) 0:04:12.446 ******* 2019-12-08T12:39:27.443 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-08T12:39:27.445 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.480 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.064) 0:04:12.511 ******* 2019-12-08T12:39:27.507 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-08T12:39:27.509 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.547 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.067) 0:04:12.578 ******* 2019-12-08T12:39:27.564 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-08T12:39:27.566 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-08T12:39:27.603 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.055) 0:04:12.634 ******* 2019-12-08T12:39:27.609 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-08T12:39:27.611 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.648 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.045) 0:04:12.679 ******* 2019-12-08T12:39:27.674 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-08T12:39:27.675 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.714 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.064) 0:04:12.743 ******* 2019-12-08T12:39:27.740 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-08T12:39:27.742 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.780 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.067) 0:04:12.811 ******* 2019-12-08T12:39:27.806 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-08T12:39:27.808 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:27.843 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.063) 0:04:12.874 ******* 2019-12-08T12:39:27.942 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-08T12:39:27.944 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-08T12:39:27.980 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:27 +0000 (0:00:00.136) 0:04:13.011 ******* 2019-12-08T12:39:28.185 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-08T12:39:28.186 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-08T12:39:28.232 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.251) 0:04:13.262 ******* 2019-12-08T12:39:28.311 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-08T12:39:28.313 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-08T12:39:28.356 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.124) 0:04:13.386 ******* 2019-12-08T12:39:28.399 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-08T12:39:28.400 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.437 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.081) 0:04:13.468 ******* 2019-12-08T12:39:28.465 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.501 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.063) 0:04:13.532 ******* 2019-12-08T12:39:28.528 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-08T12:39:28.529 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.569 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.067) 0:04:13.599 ******* 2019-12-08T12:39:28.574 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-08T12:39:28.575 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-08T12:39:28.620 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.045) 0:04:13.645 ******* TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.660 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.045) 0:04:13.691 ******* 2019-12-08T12:39:28.699 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-08T12:39:28.700 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.743 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.082) 0:04:13.773 ******* 2019-12-08T12:39:28.769 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-08T12:39:28.771 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.808 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.065) 0:04:13.839 ******* 2019-12-08T12:39:28.835 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-08T12:39:28.836 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.872 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.063) 0:04:13.903 ******* 2019-12-08T12:39:28.898 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-08T12:39:28.900 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.935 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.062) 0:04:13.966 ******* 2019-12-08T12:39:28.961 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-08T12:39:28.962 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:28.999 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:28 +0000 (0:00:00.064) 0:04:14.030 ******* 2019-12-08T12:39:29.006 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-08T12:39:29.007 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-08T12:39:29.045 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.046) 0:04:14.076 ******* 2019-12-08T12:39:29.051 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-08T12:39:29.052 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.090 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.121 ******* 2019-12-08T12:39:29.096 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-08T12:39:29.099 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.136 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.045) 0:04:14.166 ******* 2019-12-08T12:39:29.141 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-08T12:39:29.142 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.181 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.211 ******* 2019-12-08T12:39:29.188 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-08T12:39:29.189 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.227 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.046) 0:04:14.258 ******* 2019-12-08T12:39:29.233 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-08T12:39:29.235 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.276 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.048) 0:04:14.306 ******* 2019-12-08T12:39:29.281 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-08T12:39:29.283 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.320 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.043) 0:04:14.350 ******* 2019-12-08T12:39:29.327 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-08T12:39:29.330 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.364 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.395 ******* 2019-12-08T12:39:29.371 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-08T12:39:29.371 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.409 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.440 ******* 2019-12-08T12:39:29.413 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-08T12:39:29.414 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.452 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.043) 0:04:14.483 ******* 2019-12-08T12:39:29.457 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-08T12:39:29.459 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.495 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.043) 0:04:14.526 ******* 2019-12-08T12:39:29.499 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-08T12:39:29.500 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.538 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.042) 0:04:14.568 ******* 2019-12-08T12:39:29.544 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-08T12:39:29.546 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.601 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.062) 0:04:14.631 ******* 2019-12-08T12:39:29.610 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-08T12:39:29.611 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.651 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.050) 0:04:14.682 ******* 2019-12-08T12:39:29.665 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.696 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.727 ******* 2019-12-08T12:39:29.702 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-08T12:39:29.702 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.740 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.771 ******* 2019-12-08T12:39:29.746 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-08T12:39:29.748 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.784 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.043) 0:04:14.815 ******* 2019-12-08T12:39:29.789 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-08T12:39:29.792 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.829 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.045) 0:04:14.860 ******* 2019-12-08T12:39:29.836 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-08T12:39:29.838 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.882 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.045) 0:04:14.906 ******* TASK [testnode : Stop collectl] ************************************************ skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.919 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.044) 0:04:14.950 ******* 2019-12-08T12:39:29.925 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-08T12:39:29.927 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:29.963 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:29 +0000 (0:00:00.043) 0:04:14.994 ******* 2019-12-08T12:39:29.969 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-08T12:39:29.969 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.009 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.044) 0:04:15.038 ******* 2019-12-08T12:39:30.016 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-08T12:39:30.017 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.053 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.044) 0:04:15.083 ******* 2019-12-08T12:39:30.058 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-08T12:39:30.060 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.097 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.043) 0:04:15.127 ******* 2019-12-08T12:39:30.103 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-08T12:39:30.104 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.146 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.049) 0:04:15.177 ******* 2019-12-08T12:39:30.152 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-08T12:39:30.153 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.191 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.044) 0:04:15.221 ******* 2019-12-08T12:39:30.197 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-08T12:39:30.198 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.239 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.048) 0:04:15.269 ******* 2019-12-08T12:39:30.246 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:39:30.248 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-08T12:39:30.284 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.045) 0:04:15.315 ******* 2019-12-08T12:39:30.290 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-08T12:39:30.292 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.335 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.045) 0:04:15.360 ******* 2019-12-08T12:39:30.337 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi129.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.348 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-08T12:39:30.348 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.359 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-08T12:39:30.361 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi129.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.398 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.068) 0:04:15.429 ******* 2019-12-08T12:39:30.407 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.453 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.049) 0:04:15.479 ******* 2019-12-08T12:39:30.454 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-08T12:39:30.455 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.492 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.043) 0:04:15.522 ******* 2019-12-08T12:39:30.518 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-08T12:39:30.520 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.557 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.065) 0:04:15.587 ******* 2019-12-08T12:39:30.606 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-08T12:39:30.607 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003882", "end": "2019-12-08 12:39:30.579223", "rc": 0, "start": "2019-12-08 12:39:30.575341", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-08T12:39:30.608 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.650 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.093) 0:04:15.681 ******* 2019-12-08T12:39:30.707 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.056) 0:04:15.737 ******* 2019-12-08T12:39:30.713 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-08T12:39:30.714 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.760 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.052) 0:04:15.790 ******* 2019-12-08T12:39:30.766 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-08T12:39:30.768 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.810 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.050) 0:04:15.841 ******* 2019-12-08T12:39:30.816 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-08T12:39:30.820 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.859 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.048) 0:04:15.889 ******* 2019-12-08T12:39:30.866 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.904 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.045) 0:04:15.935 ******* 2019-12-08T12:39:30.910 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-08T12:39:30.911 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.951 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.044) 0:04:15.979 ******* 2019-12-08T12:39:30.954 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-08T12:39:30.956 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:30.993 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:30 +0000 (0:00:00.044) 0:04:16.023 ******* 2019-12-08T12:39:30.998 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-08T12:39:31.000 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.036 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.043) 0:04:16.066 ******* 2019-12-08T12:39:31.044 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-08T12:39:31.046 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.083 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.046) 0:04:16.113 ******* 2019-12-08T12:39:31.109 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:39:31.110 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-08T12:39:31.159 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.075) 0:04:16.189 ******* 2019-12-08T12:39:31.216 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-08T12:39:31.218 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.227 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-08T12:39:31.238 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.249 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.250 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.271 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.272 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.314 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.155) 0:04:16.345 ******* 2019-12-08T12:39:31.461 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-08T12:39:31.461 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003785", "end": "2019-12-08 12:39:31.434872", "rc": 0, "start": "2019-12-08 12:39:31.431087", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-08T12:39:31.502 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.187) 0:04:16.532 ******* 2019-12-08T12:39:31.538 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-08T12:39:31.540 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.579 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.075) 0:04:16.608 ******* 2019-12-08T12:39:31.614 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.036) 0:04:16.645 ******* 2019-12-08T12:39:31.619 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-08T12:39:31.625 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item={u'block_used': 13040430, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284054, u'size_available': 930955485184, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284054, "block_size": 4096, "block_total": 240324484, "block_used": 13040430, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955485184, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-08T12:39:31.660 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.045) 0:04:16.690 ******* 2019-12-08T12:39:31.666 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-08T12:39:31.667 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.704 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.044) 0:04:16.735 ******* 2019-12-08T12:39:31.730 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-08T12:39:31.732 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.768 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.063) 0:04:16.798 ******* 2019-12-08T12:39:31.795 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-08T12:39:31.798 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:31.832 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:31 +0000 (0:00:00.063) 0:04:16.862 ******* 2019-12-08T12:39:32.004 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-08T12:39:32.006 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011911", "end": "2019-12-08 12:39:31.970174", "rc": 0, "start": "2019-12-08 12:39:31.958263", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:32.042 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.210) 0:04:17.072 ******* 2019-12-08T12:39:32.090 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-08T12:39:32.091 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'block_used': 13040430, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284054, u'size_available': 930955485184, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284054, "block_size": 4096, "block_total": 240324484, "block_used": 13040430, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955485184, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:32.134 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.092) 0:04:17.165 ******* 2019-12-08T12:39:32.160 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-08T12:39:32.199 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.065) 0:04:17.230 ******* 2019-12-08T12:39:32.238 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-08T12:39:32.240 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XGTA', u'wwn-0x5000c5009292d832'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XGTA', u'wwn': u'0x5000c5009292d832', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XGTA-part1', u'wwn-0x5000c5009292d832-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGTA", "wwn-0x5000c5009292d832"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGTA-part1", "wwn-0x5000c5009292d832-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XGTA", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-08T12:39:32.241 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009292d832"}}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:32.337 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-08T12:39:32.378 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.178) 0:04:17.408 ******* 2019-12-08T12:39:32.420 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-08T12:39:32.422 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item={u'block_used': 13040430, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284054, u'size_available': 930955485184, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284054, "block_size": 4096, "block_total": 240324484, "block_used": 13040430, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955485184, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-08T12:39:32.483 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.105) 0:04:17.514 ******* 2019-12-08T12:39:32.801 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-08T12:39:32.803 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.012006", "end": "2019-12-08 12:39:32.779876", "rc": 0, "start": "2019-12-08 12:39:32.767870", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:32.842 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.358) 0:04:17.872 ******* 2019-12-08T12:39:32.888 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-08T12:39:32.890 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'block_used': 13040430, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284054, u'size_available': 930955485184, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284054, "block_size": 4096, "block_total": 240324484, "block_used": 13040430, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955485184, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:32.930 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.088) 0:04:17.960 ******* 2019-12-08T12:39:32.959 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-08T12:39:32.996 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:32 +0000 (0:00:00.066) 0:04:18.027 ******* 2019-12-08T12:39:33.035 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-08T12:39:33.036 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTWX', u'wwn-0x5000c5009293ee09'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XTWX', u'wwn': u'0x5000c5009293ee09', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1', u'wwn-0x5000c5009293ee09-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX", "wwn-0x5000c5009293ee09"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1", "wwn-0x5000c5009293ee09-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTWX", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-08T12:39:33.038 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009293ee09"}}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:34.143 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-hpHSq8-YUCa-ycRb-oojc-Rqog-tJAg-nZ2fSk', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300HY400BGN', u'nvme-nvme.8086-43564654363233333030485934303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.586351", "end": "2019-12-08 12:39:34.106946", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-hpHSq8-YUCa-ycRb-oojc-Rqog-tJAg-nZ2fSk", "nvme-INTEL_SSDPEDMD400G4_CVFT623300HY400BGN", "nvme-nvme.8086-43564654363233333030485934303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:32.520 2019-12-08T12:39:34.145 INFO:teuthology.task.ansible.out:595", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:34.845 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-TC2w1J-b6Dn-ndh8-f4XZ-BUlj-JvEP-D0KFKH', u'nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN', u'nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.485627", "end": "2019-12-08 12:39:34.810012", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-TC2w1J-b6Dn-ndh8-f4XZ-BUlj-JvEP-D0KFKH", "nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN", "nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:33.324 2019-12-08T12:39:34.846 INFO:teuthology.task.ansible.out:385", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:35.515 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNrQGwYLZRugUcDIBpVJvXRzZipnPIlzny'], u'uuids': [u'cb760773-b6b8-47cc-ba70-430aa8e5b992']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.065112", "end": "2019-12-08 12:39:35.484079", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNrQGwYLZRugUcDIBpVJvXRzZipnPIlzny"], "labels": [], "masters": [], "uuids": ["cb760773-b6b8-47cc-ba70-430aa8e5b992"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:34.418967", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:36.252 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8ud19GeVqptPPGOK3Fwa1MUEQxLCp3T40nD'], u'uuids': [u'1811afee-f21f-4974-9323-24af65b64478']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.091241", "end": "2019-12-08 12:39:36.209084", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8ud19GeVqptPPGOK3Fwa1MUEQxLCp3T40nD"], "labels": [], "masters": [], "uuids": ["1811afee-f21f-4974-9323-24af65b64478"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:35.117843", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:36.900 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRdhCxQgciAndzZhlmbejN30d8AfGXP1u'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.083611", "end": "2019-12-08 12:39:36.866995", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRdhCxQgciAndzZhlmbejN30d8AfGXP1u"], "labels": [], "masters": [], "uuids": []}, "model": null, "par 2019-12-08T12:39:36.902 INFO:teuthology.task.ansible.out:titions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:35.783384", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:37.743 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udKXaam7V9ujI3lWRXihvFJbHuuld3IaAh'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.156585", "end": "2019-12-08 12:39:37.709773", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udKXaam7V9ujI3lWRXihvFJbHuuld3IaAh"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:36.553188", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:38.269 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRiUzmkH2FaEzmyF6JfPK8zxtWBETQY2L'], u'uuids': [u'd62c1b3a-b74d-440c-8395-f524bd1ae643']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.056519", "end": "2019-12-08 12:39:38.235586", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRiUzmkH2FaEzmyF6JfPK8zxtWBETQY2L"], "labels": [], "masters": [], "uuids": ["d62c1b3a-b74d-440c-8395-f524bd1ae643"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:37.179067", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:39.112 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udrRw1adKsX749QRYql09yY4fwEugFtSwt'], u'uuids': [u'b30d43b3-b8d4-4b05-a1de-6a052ad3f97f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.060685", "end": "2019-12-08 12:39:39.076967", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udrRw1adKsX749QRYql09yY4fwEugFtSwt"], "labels": [], "masters": [], "uuids": ["b30d43b3-b8d4-4b05-a1de-6a052ad3f97f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:38.016282", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:39.673 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNHo9OgVIEqYtrfgCMTF9U04kdwnT4oHW7'], u'uuids': [u'54a9756f-ac4c-4a02-8110-515d1c859f0a']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.081045", "end": "2019-12-08 12:39:39.643821", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNHo9OgVIEqYtrfgCMTF9U04kdwnT4oHW7"], "labels": [], "masters": [], "uuids": ["54a9756f-ac4c-4a02-8110-515d1c859f0a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:38.562776", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:40.472 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udm5FhnlS2aTn3TpdJ517rXH59gOuUc33v'], u'uuids': [u'bb8cf6d4-ded7-434d-b32f-f0cade313693']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.067084", "end": "2019-12-08 12:39:40.434332", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udm5FhnlS2aTn3TpdJ517rXH59gOuUc33v"], "labels": [], "masters": [], "uuids": ["bb8cf6d4-ded7-434d-b32f-f0cade313693"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:39.367248", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:41.054 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNsDHxYYFQlPqUlje5Qlc9ztLPyuc4bMB3'], u'uuids': [u'77a8ef1f-e4fe-461a-93f9-40b7e47e4898']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.067044", "end": "2019-12-08 12:39:41.018681", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNsDHxYYFQlPqUlje5Qlc9ztLPyuc4bMB3"], "labels": [], "masters": [], "uuids": ["77a8ef1f-e4fe-461a-93f9-40b7e47e4898"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:39.951637", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:41.098 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:41 +0000 (0:00:08.101) 0:04:26.128 ******* 2019-12-08T12:39:41.149 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XGTA', u'wwn-0x5000c5009292d832'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XGTA', u'wwn': u'0x5000c5009292d832', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XGTA-part1', u'wwn-0x5000c5009292d832-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGTA", "wwn-0x5000c5009292d832"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGTA-part1", "wwn-0x5000c5009292d832-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": " 2019-12-08T12:39:41.151 INFO:teuthology.task.ansible.out:512", "serial": "Z1W5XGTA", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009292d832"}}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:41.510 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-hpHSq8-YUCa-ycRb-oojc-Rqog-tJAg-nZ2fSk', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300HY400BGN', u'nvme-nvme.8086-43564654363233333030485934303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.065251", "end": "2019-12-08 12:39:41.488637", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-hpHSq8-YUCa-ycRb-oojc-Rqog-tJAg-nZ2fSk", "nvme-INTEL_SSDPEDMD400G4_CVFT623300HY400BGN", "nvme-nvme.8086-43564654363233333030485934303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:41.423386", "stderr": 2019-12-08T12:39:41.511 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0587386 s, 2.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0587386 s, 2.0 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:41.833 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-08T12:39:41.836 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udu76EB2H58if9PtO5ayQpElVyB7Kw7wIk'], u'uuids': [u'2dc39490-f56d-46a1-914a-7e5efe99a691']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.054111", "end": "2019-12-08 12:39:41.802111", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udu76EB2H58if9PtO5ayQpElVyB7Kw7wIk"], "labels": [], "masters": [], "uuids": ["2dc39490-f56d-46a1-914a-7e5efe99a691"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:40.748000", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-08T12:39:41.890 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:41 +0000 (0:00:00.792) 0:04:26.920 ******* 2019-12-08T12:39:41.926 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-08T12:39:41.927 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTWX', u'wwn-0x5000c5009293ee09'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XTWX', u'wwn': u'0x5000c5009293ee09', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1', u'wwn-0x5000c5009293ee09-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX", "wwn-0x5000c5009293ee09"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTWX-part1", "wwn-0x5000c5009293ee09-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTWX", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-08T12:39:41.929 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009293ee09"}}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:41.961 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNrQGwYLZRugUcDIBpVJvXRzZipnPIlzny'], u'uuids': [u'cb760773-b6b8-47cc-ba70-430aa8e5b992']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.169572", "end": "2019-12-08 12:39:41.932813", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNrQGwYLZRugUcDIBpVJvXRzZipnPIlzny"], "labels": [], "masters": [], "uuids": ["cb760773-b6b8-47cc-ba70-430aa8e5b992"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:41.763241", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.163523 s, 705 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.163523 s, 705 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:42.274 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-TC2w1J-b6Dn-ndh8-f4XZ-BUlj-JvEP-D0KFKH', u'nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN', u'nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.062821", "end": "2019-12-08 12:39:42.246691", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-TC2w1J-b6Dn-ndh8-f4XZ-BUlj-JvEP-D0KFKH", "nvme-INTEL_SSDPEDMD400G4_CVFT6246000E400BGN", "nvme-nvme.8086-43564654363234363030304534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:42.183870", "stderr": 2019-12-08T12:39:42.276 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0567332 s, 2.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0567332 s, 2.0 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:42.410 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRdhCxQgciAndzZhlmbejN30d8AfGXP1u'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.170321", "end": "2019-12-08 12:39:42.387280", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRdhCxQgciAndzZhlmbejN30d8AfGXP1u"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:42.216959", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.164198 s, 702 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.164198 s, 702 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:42.722 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8ud19GeVqptPPGOK3Fwa1MUEQxLCp3T40nD'], u'uuids': [u'1811afee-f21f-4974-9323-24af65b64478']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.182258", "end": "2019-12-08 12:39:42.691792", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8ud19GeVqptPPGOK3Fwa1MUEQxLCp3T40nD"], "labels": [], "masters": [], "uuids": ["1811afee-f21f-4974-9323-24af65b64478"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:42.509534", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.177241 s, 651 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.177241 s, 651 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:42.895 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRiUzmkH2FaEzmyF6JfPK8zxtWBETQY2L'], u'uuids': [u'd62c1b3a-b74d-440c-8395-f524bd1ae643']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.180767", "end": "2019-12-08 12:39:42.873248", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNRiUzmkH2FaEzmyF6JfPK8zxtWBETQY2L"], "labels": [], "masters": [], "uuids": ["d62c1b3a-b74d-440c-8395-f524bd1ae643"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:42.692481", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.17462 s, 661 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.17462 s, 661 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:43.249 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udKXaam7V9ujI3lWRXihvFJbHuuld3IaAh'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.196619", "end": "2019-12-08 12:39:43.205006", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udKXaam7V9ujI3lWRXihvFJbHuuld3IaAh"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:43.008387", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.19008 s, 607 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.19008 s, 607 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:43.407 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNHo9OgVIEqYtrfgCMTF9U04kdwnT4oHW7'], u'uuids': [u'54a9756f-ac4c-4a02-8110-515d1c859f0a']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.198021", "end": "2019-12-08 12:39:43.378070", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNHo9OgVIEqYtrfgCMTF9U04kdwnT4oHW7"], "labels": [], "masters": [], "uuids": ["54a9756f-ac4c-4a02-8110-515d1c859f0a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:43.180049", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.191388 s, 603 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.191388 s, 603 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:43.749 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udrRw1adKsX749QRYql09yY4fwEugFtSwt'], u'uuids': [u'b30d43b3-b8d4-4b05-a1de-6a052ad3f97f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.190742", "end": "2019-12-08 12:39:43.718661", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udrRw1adKsX749QRYql09yY4fwEugFtSwt"], "labels": [], "masters": [], "uuids": ["b30d43b3-b8d4-4b05-a1de-6a052ad3f97f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:43.527919", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.184607 s, 625 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.184607 s, 625 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:43.878 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNsDHxYYFQlPqUlje5Qlc9ztLPyuc4bMB3'], u'uuids': [u'77a8ef1f-e4fe-461a-93f9-40b7e47e4898']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300HY400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.172622", "end": "2019-12-08 12:39:43.854541", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rbExXJkMbZQV8QQnwFY2Ey9Z0V75ctuNsDHxYYFQlPqUlje5Qlc9ztLPyuc4bMB3"], "labels": [], "masters": [], "uuids": ["77a8ef1f-e4fe-461a-93f9-40b7e47e4898"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300HY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:43.681919", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.166076 s, 695 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.166076 s, 695 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:43.926 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:43 +0000 (0:00:02.036) 0:04:28.957 ******* 2019-12-08T12:39:44.214 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udm5FhnlS2aTn3TpdJ517rXH59gOuUc33v'], u'uuids': [u'bb8cf6d4-ded7-434d-b32f-f0cade313693']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.173312", "end": "2019-12-08 12:39:44.180594", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udm5FhnlS2aTn3TpdJ517rXH59gOuUc33v"], "labels": [], "masters": [], "uuids": ["bb8cf6d4-ded7-434d-b32f-f0cade313693"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:44.007282", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.167737 s, 688 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.167737 s, 688 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:44.350 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-08T12:39:44.352 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.093313", "end": "2019-12-08 12:39:44.321585", "rc": 0, "start": "2019-12-08 12:39:44.228272", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:44.397 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:44 +0000 (0:00:00.470) 0:04:29.428 ******* 2019-12-08T12:39:44.720 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-08T12:39:44.721 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udu76EB2H58if9PtO5ayQpElVyB7Kw7wIk'], u'uuids': [u'2dc39490-f56d-46a1-914a-7e5efe99a691']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT6246000E400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.198911", "end": "2019-12-08 12:39:44.682980", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-kka6CeeSR0ZctxeWTh2AObusfs3Zp8udu76EB2H58if9PtO5ayQpElVyB7Kw7wIk"], "labels": [], "masters": [], "uuids": ["2dc39490-f56d-46a1-914a-7e5efe99a691"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT6246000E400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-08 12:39:44.484069", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.192201 s, 600 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.192201 s, 600 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:44.788 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:44 +0000 (0:00:00.391) 0:04:29.819 ******* 2019-12-08T12:39:44.798 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-08T12:39:44.799 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.053257", "end": "2019-12-08 12:39:44.744853", "rc": 0, "start": "2019-12-08 12:39:44.691596", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:44.840 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:44 +0000 (0:00:00.051) 0:04:29.871 ******* 2019-12-08T12:39:44.867 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-08T12:39:44.908 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:44 +0000 (0:00:00.067) 0:04:29.938 ******* 2019-12-08T12:39:44.935 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:44.976 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:44 +0000 (0:00:00.068) 0:04:30.007 ******* 2019-12-08T12:39:45.003 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-08T12:39:45.004 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.042 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.066) 0:04:30.073 ******* 2019-12-08T12:39:45.069 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-08T12:39:45.071 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.109 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.066) 0:04:30.139 ******* 2019-12-08T12:39:45.146 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-08T12:39:45.148 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.183 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.074) 0:04:30.214 ******* 2019-12-08T12:39:45.189 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-08T12:39:45.189 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.079541", "end": "2019-12-08 12:39:45.145571", "rc": 0, "start": "2019-12-08 12:39:45.066030", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:45.231 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.048) 0:04:30.262 ******* 2019-12-08T12:39:45.232 INFO:teuthology.task.ansible.out: 2019-12-08T12:39:45.514 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-08T12:39:45.516 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:39:45.562 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.330) 0:04:30.593 ******* 2019-12-08T12:39:45.613 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-08T12:39:45.613 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={u'block_used': 13040430, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284054, u'size_available': 930955485184, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284054, "block_size": 4096, "block_total": 240324484, "block_used": 13040430, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955485184, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.652 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.089) 0:04:30.682 ******* 2019-12-08T12:39:45.658 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-08T12:39:45.659 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.055840", "end": "2019-12-08 12:39:45.601905", "rc": 0, "start": "2019-12-08 12:39:45.546065", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:39:45.705 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.053) 0:04:30.735 ******* 2019-12-08T12:39:45.711 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-08T12:39:45.713 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.750 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.045) 0:04:30.781 ******* 2019-12-08T12:39:45.756 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-08T12:39:45.802 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.051) 0:04:30.832 ******* 2019-12-08T12:39:45.808 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:39:45.810 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.847 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.045) 0:04:30.878 ******* 2019-12-08T12:39:45.854 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-08T12:39:45.856 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.896 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.049) 0:04:30.927 ******* 2019-12-08T12:39:45.901 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:39:45.902 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.941 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.044) 0:04:30.971 ******* 2019-12-08T12:39:45.948 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-08T12:39:45.949 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:45.989 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:45 +0000 (0:00:00.048) 0:04:31.020 ******* 2019-12-08T12:39:45.995 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-08T12:39:45.996 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.034 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.044) 0:04:31.064 ******* 2019-12-08T12:39:46.042 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-08T12:39:46.044 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.083 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.048) 0:04:31.113 ******* 2019-12-08T12:39:46.088 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-08T12:39:46.089 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.127 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.044) 0:04:31.157 ******* 2019-12-08T12:39:46.131 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-08T12:39:46.133 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.173 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.046) 0:04:31.204 ******* 2019-12-08T12:39:46.179 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-08T12:39:46.180 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.219 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.045) 0:04:31.249 ******* 2019-12-08T12:39:46.499 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-08T12:39:46.501 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-08T12:39:46.538 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.319) 0:04:31.569 ******* 2019-12-08T12:39:46.586 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-08T12:39:46.587 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={u'block_used': 13040430, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284054, u'size_available': 930955485184, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284054, "block_size": 4096, "block_total": 240324484, "block_used": 13040430, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955485184, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.625 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.086) 0:04:31.656 ******* 2019-12-08T12:39:46.652 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-08T12:39:46.654 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.701 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.075) 0:04:31.731 ******* 2019-12-08T12:39:46.738 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:39:46.740 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.777 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.075) 0:04:31.807 ******* 2019-12-08T12:39:46.803 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:39:46.805 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.843 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.066) 0:04:31.874 ******* 2019-12-08T12:39:46.870 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-08T12:39:46.871 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.909 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.065) 0:04:31.939 ******* 2019-12-08T12:39:46.946 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-08T12:39:46.947 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:46.985 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:46 +0000 (0:00:00.075) 0:04:32.015 ******* 2019-12-08T12:39:47.023 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-08T12:39:47.024 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create volume group(s)] *************************************** changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-08T12:39:47.062 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:47 +0000 (0:00:00.076) 0:04:32.092 ******* 2019-12-08T12:39:47.106 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:47 +0000 (0:00:00.044) 0:04:32.136 ******* 2019-12-08T12:39:47.604 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-08T12:39:47.648 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:47 +0000 (0:00:00.537) 0:04:32.674 ******* 2019-12-08T12:39:47.876 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:48.167 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:48.354 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:48.620 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:48.831 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:49.102 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:49.280 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:49.575 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:49.777 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:49.814 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:49 +0000 (0:00:02.170) 0:04:34.845 ******* 2019-12-08T12:39:50.043 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-08T12:39:50.084 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:50 +0000 (0:00:00.270) 0:04:35.115 ******* 2019-12-08T12:39:50.154 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-08T12:39:50.155 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-08T12:39:50.199 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:50 +0000 (0:00:00.114) 0:04:35.229 ******* 2019-12-08T12:39:50.237 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-08T12:39:50.238 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:50.417 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-08T12:39:50.418 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-08T12:39:50.457 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:50 +0000 (0:00:00.257) 0:04:35.487 ******* 2019-12-08T12:39:50.508 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-08T12:39:50.509 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:50.553 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:50.819 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:50.873 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:51.112 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:51.168 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:51.422 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:51.487 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:51.529 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:51 +0000 (0:00:01.071) 0:04:36.559 ******* 2019-12-08T12:39:51.694 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-08T12:39:51.738 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:51 +0000 (0:00:00.209) 0:04:36.768 ******* 2019-12-08T12:39:51.841 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-08T12:39:51.841 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-08T12:39:51.880 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:51 +0000 (0:00:00.142) 0:04:36.911 ******* 2019-12-08T12:39:51.918 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-08T12:39:51.919 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-08T12:39:51.959 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:51 +0000 (0:00:00.078) 0:04:36.990 ******* 2019-12-08T12:39:52.049 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-08T12:39:52.049 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-08T12:39:52.113 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:52 +0000 (0:00:00.153) 0:04:37.143 ******* 2019-12-08T12:39:52.150 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-08T12:39:52.151 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-08T12:39:52.188 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:52 +0000 (0:00:00.075) 0:04:37.219 ******* 2019-12-08T12:39:52.641 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-08T12:39:52.643 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true} 2019-12-08T12:39:52.683 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:52 +0000 (0:00:00.494) 0:04:37.713 ******* 2019-12-08T12:39:52.688 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true} 2019-12-08T12:39:52.728 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:52 +0000 (0:00:00.045) 0:04:37.759 ******* 2019-12-08T12:39:53.409 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-08T12:39:53.410 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-08T12:39:53.448 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.720) 0:04:38.479 ******* 2019-12-08T12:39:53.455 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-08T12:39:53.499 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.050) 0:04:38.529 ******* 2019-12-08T12:39:53.504 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-08T12:39:53.506 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.546 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.047) 0:04:38.577 ******* 2019-12-08T12:39:53.554 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.592 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.045) 0:04:38.623 ******* 2019-12-08T12:39:53.598 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-08T12:39:53.599 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.639 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.046) 0:04:38.670 ******* 2019-12-08T12:39:53.645 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.686 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.046) 0:04:38.716 ******* 2019-12-08T12:39:53.691 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.730 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.044) 0:04:38.761 ******* 2019-12-08T12:39:53.736 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.776 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.045) 0:04:38.806 ******* 2019-12-08T12:39:53.785 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-08T12:39:53.786 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.826 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.049) 0:04:38.856 ******* 2019-12-08T12:39:53.832 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.880 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.054) 0:04:38.911 ******* 2019-12-08T12:39:53.886 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-08T12:39:53.888 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.931 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.051) 0:04:38.962 ******* 2019-12-08T12:39:53.938 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:53.977 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:53 +0000 (0:00:00.045) 0:04:39.008 ******* 2019-12-08T12:39:53.985 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.023 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.045) 0:04:39.053 ******* 2019-12-08T12:39:54.032 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.070 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.047) 0:04:39.100 ******* 2019-12-08T12:39:54.076 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-08T12:39:54.077 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.114 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.044) 0:04:39.145 ******* 2019-12-08T12:39:54.120 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.181 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.066) 0:04:39.211 ******* 2019-12-08T12:39:54.604 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-08T12:39:54.605 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-08T12:39:54.643 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.462) 0:04:39.673 ******* 2019-12-08T12:39:54.650 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-08T12:39:54.707 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.064) 0:04:39.738 ******* 2019-12-08T12:39:54.713 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-08T12:39:54.715 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.770 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.062) 0:04:39.800 ******* 2019-12-08T12:39:54.777 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.817 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.046) 0:04:39.847 ******* 2019-12-08T12:39:54.841 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.863 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.046) 0:04:39.893 ******* 2019-12-08T12:39:54.868 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:54.908 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:54 +0000 (0:00:00.044) 0:04:39.938 ******* 2019-12-08T12:39:55.815 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-08T12:39:55.816 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-08T12:39:55.855 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:55 +0000 (0:00:00.947) 0:04:40.885 ******* 2019-12-08T12:39:55.862 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-08T12:39:55.902 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:55 +0000 (0:00:00.046) 0:04:40.932 ******* 2019-12-08T12:39:55.908 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-08T12:39:55.910 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:55.949 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:55 +0000 (0:00:00.047) 0:04:40.979 ******* 2019-12-08T12:39:55.958 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:56.001 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:55 +0000 (0:00:00.051) 0:04:41.031 ******* 2019-12-08T12:39:56.452 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-08T12:39:56.456 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "-.mount systemd-journald.socket ntpdate.service basic.target sntp.service tmp.mount system.slice syslog.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target multi-user.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-08T12:39:56.457 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-08T12:39:56.459 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-08T12:39:56.502 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:56 +0000 (0:00:00.501) 0:04:41.532 ******* 2019-12-08T12:39:56.511 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket -.mount ntpdate.service syslog.target basic.target tmp.mount system.slice sntp.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target chronyd.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-08T12:39:56.512 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-08T12:39:56.514 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-08T12:39:56.552 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:56 +0000 (0:00:00.049) 0:04:41.582 ******* 2019-12-08T12:39:57.448 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-08T12:39:57.449 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-08T12:39:57.493 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:57 +0000 (0:00:00.941) 0:04:42.524 ******* 2019-12-08T12:39:57.504 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-08T12:39:57.559 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:57 +0000 (0:00:00.065) 0:04:42.589 ******* 2019-12-08T12:39:57.565 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-08T12:39:57.565 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:57.605 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:57 +0000 (0:00:00.045) 0:04:42.636 ******* 2019-12-08T12:39:57.611 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:57.651 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:57 +0000 (0:00:00.045) 0:04:42.681 ******* 2019-12-08T12:39:57.659 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-08T12:39:57.661 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:57.698 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:57 +0000 (0:00:00.047) 0:04:42.729 ******* 2019-12-08T12:39:57.704 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:39:57.766 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:57 +0000 (0:00:00.067) 0:04:42.797 ******* 2019-12-08T12:39:58.026 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-08T12:39:58.027 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-08T12:39:58.065 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:58 +0000 (0:00:00.299) 0:04:43.096 ******* 2019-12-08T12:39:58.104 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-08T12:39:58.141 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:58 +0000 (0:00:00.076) 0:04:43.172 ******* 2019-12-08T12:39:58.957 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-08T12:39:58.959 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-08T12:39:58.995 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:58 +0000 (0:00:00.853) 0:04:44.025 ******* 2019-12-08T12:39:59.074 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-08T12:39:59.110 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:39:59 +0000 (0:00:00.115) 0:04:44.141 ******* 2019-12-08T12:40:00.782 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-08T12:40:00.819 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:00 +0000 (0:00:01.708) 0:04:45.849 ******* 2019-12-08T12:40:00.846 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-08T12:40:00.847 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:00.888 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:00 +0000 (0:00:00.068) 0:04:45.918 ******* 2019-12-08T12:40:00.930 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:00 +0000 (0:00:00.042) 0:04:45.960 ******* 2019-12-08T12:40:00.936 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-08T12:40:00.938 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-08T12:40:00.976 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:00 +0000 (0:00:00.046) 0:04:46.007 ******* 2019-12-08T12:40:00.983 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-08T12:40:00.986 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:01.025 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.048) 0:04:46.055 ******* 2019-12-08T12:40:01.054 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-08T12:40:01.056 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-08T12:40:01.111 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.084) 0:04:46.140 ******* 2019-12-08T12:40:01.303 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-08T12:40:01.305 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-08T12:40:01.343 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.234) 0:04:46.374 ******* 2019-12-08T12:40:01.372 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-08T12:40:01.374 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:01.410 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.066) 0:04:46.440 ******* 2019-12-08T12:40:01.417 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-08T12:40:01.418 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-08T12:40:01.470 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.059) 0:04:46.500 ******* 2019-12-08T12:40:01.477 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-08T12:40:01.478 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-08T12:40:01.522 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.052) 0:04:46.553 ******* 2019-12-08T12:40:01.528 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-08T12:40:01.530 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:01.570 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.047) 0:04:46.601 ******* 2019-12-08T12:40:01.578 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-08T12:40:01.580 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:01.620 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.049) 0:04:46.650 ******* 2019-12-08T12:40:01.627 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-08T12:40:01.629 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-08T12:40:01.667 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.046) 0:04:46.697 ******* 2019-12-08T12:40:01.693 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-08T12:40:01.695 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:01.734 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:01 +0000 (0:00:00.067) 0:04:46.765 ******* 2019-12-08T12:40:01.967 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-08T12:40:01.972 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-08T12:40:02.007 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.272) 0:04:47.037 ******* 2019-12-08T12:40:02.036 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-08T12:40:02.075 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.068) 0:04:47.106 ******* 2019-12-08T12:40:02.359 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi129.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-08T12:40:02.423 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.347) 0:04:47.453 ******* 2019-12-08T12:40:02.430 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-08T12:40:02.471 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.048) 0:04:47.501 ******* 2019-12-08T12:40:02.476 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-08T12:40:02.478 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.517 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.045) 0:04:47.547 ******* 2019-12-08T12:40:02.527 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.567 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.050) 0:04:47.598 ******* 2019-12-08T12:40:02.574 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-08T12:40:02.576 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.578 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.622 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.054) 0:04:47.652 ******* 2019-12-08T12:40:02.629 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.630 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.631 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi181.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.633 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.691 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.069) 0:04:47.722 ******* 2019-12-08T12:40:02.698 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-08T12:40:02.698 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.700 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.701 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.703 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi129.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.750 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.058) 0:04:47.780 ******* 2019-12-08T12:40:02.757 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.759 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.761 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.762 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.763 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.804 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.054) 0:04:47.834 ******* 2019-12-08T12:40:02.810 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-08T12:40:02.811 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.854 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.050) 0:04:47.884 ******* 2019-12-08T12:40:02.861 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.908 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.053) 0:04:47.938 ******* 2019-12-08T12:40:02.914 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-08T12:40:02.915 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:02.955 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.047) 0:04:47.986 ******* 2019-12-08T12:40:02.961 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.003 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:02 +0000 (0:00:00.047) 0:04:48.033 ******* 2019-12-08T12:40:03.009 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-08T12:40:03.009 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.052 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.049) 0:04:48.082 ******* 2019-12-08T12:40:03.059 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.102 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.049) 0:04:48.132 ******* 2019-12-08T12:40:03.108 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-08T12:40:03.109 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.148 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.046) 0:04:48.179 ******* 2019-12-08T12:40:03.154 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.194 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.046) 0:04:48.225 ******* 2019-12-08T12:40:03.201 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-08T12:40:03.201 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.245 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.050) 0:04:48.275 ******* 2019-12-08T12:40:03.251 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.293 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.048) 0:04:48.324 ******* 2019-12-08T12:40:03.299 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-08T12:40:03.301 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.349 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.055) 0:04:48.379 ******* 2019-12-08T12:40:03.355 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.398 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.048) 0:04:48.428 ******* 2019-12-08T12:40:03.404 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-08T12:40:03.405 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.448 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.050) 0:04:48.478 ******* 2019-12-08T12:40:03.455 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.498 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.050) 0:04:48.529 ******* 2019-12-08T12:40:03.503 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-08T12:40:03.504 INFO:teuthology.task.ansible.out:skipping: [smithi129.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.546 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.047) 0:04:48.576 ******* 2019-12-08T12:40:03.553 INFO:teuthology.task.ansible.out:skipping: [smithi181.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-08T12:40:03.594 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:03 +0000 (0:00:00.047) 0:04:48.624 ******* 2019-12-08T12:40:03.899 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-08T12:40:03.900 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016510", "end": "2019-12-08 12:40:03.863738", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-08 12:40:03.847228", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:03.963 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.017017", "end": "2019-12-08 12:40:03.920844", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-08 12:40:03.903827", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:04.235 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006895", "end": "2019-12-08 12:40:04.198297", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-08 12:40:04.191402", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:04.523 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006723", "end": "2019-12-08 12:40:04.490602", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-08 12:40:04.483879", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:04.803 INFO:teuthology.task.ansible.out:ok: [smithi129.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007235", "end": "2019-12-08 12:40:04.774783", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-08 12:40:04.767548", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:04.843 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:04 +0000 (0:00:01.249) 0:04:49.874 ******* 2019-12-08T12:40:04.870 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-08T12:40:04.908 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:04 +0000 (0:00:00.064) 0:04:49.938 ******* 2019-12-08T12:40:05.224 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-08T12:40:05.225 INFO:teuthology.task.ansible.out:changed: [smithi129.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-08T12:40:05.298 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-08T12:40:05.300 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:01.008856", "end": "2019-12-08 12:40:05.232048", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-08 12:40:04.223192", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:05.581 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007480", "end": "2019-12-08 12:40:05.548716", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-08 12:40:05.541236", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:05.913 INFO:teuthology.task.ansible.out:ok: [smithi181.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008013", "end": "2019-12-08 12:40:05.882338", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-08 12:40:05.874325", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-08T12:40:05.954 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:05 +0000 (0:00:01.046) 0:04:50.985 ******* 2019-12-08T12:40:05.981 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-08T12:40:06.046 INFO:teuthology.task.ansible.out:Sunday 08 December 2019 12:40:06 +0000 (0:00:00.091) 0:04:51.076 ******* 2019-12-08T12:40:06.384 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-08T12:40:06.384 INFO:teuthology.task.ansible.out:changed: [smithi181.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-08T12:40:06.495 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2019-12-08T12:40:06.497 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2019-12-08T12:40:06.500 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2019-12-08T12:40:06.501 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi129.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 smithi181.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 Sunday 08 December 2019 12:40:06 +0000 (0:00:00.454) 0:04:51.531 ******* =============================================================================== 2019-12-08T12:40:06.503 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 34.65s users : Update authorized_keys using the keys repo --------------------- 34.00s common : nrpe - Ensure SELinux policy is up to date -------------------- 27.08s testnode : Install packages -------------------------------------------- 17.98s users : Remove revoked users ------------------------------------------- 15.03s common : Update yum cache ---------------------------------------------- 13.17s testnode : Ensure ceph packages are not present. ------------------------ 8.35s testnode : Zap all non-root disks --------------------------------------- 8.10s common : nrpe - Load SELinux policy package ----------------------------- 7.96s common : nrpe - Remove SELinux policy package --------------------------- 7.86s Gathering Facts --------------------------------------------------------- 3.39s testnode : Ensure ceph dependency packages are not present. ------------- 3.23s common : Make sure smartmontools is installed --------------------------- 3.11s ansible-managed : Add authorized keys for the ansible user. ------------- 2.88s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.62s testnode : Create logical volume(s) ------------------------------------- 2.17s testnode : Rebuild rpmdb ------------------------------------------------ 2.14s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.04s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.89s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.84s 2019-12-08T12:40:06.794 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2019-12-08T12:40:06.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:40:06.797 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi129.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:40:06.898 INFO:teuthology.orchestra.run.smithi129:> true 2019-12-08T12:40:07.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-08T12:40:07.262 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi181.front.sepia.ceph.com', 'timeout': 60} 2019-12-08T12:40:07.362 INFO:teuthology.orchestra.run.smithi181:> true 2019-12-08T12:40:07.728 INFO:teuthology.run_tasks:Running task clock... 2019-12-08T12:40:07.740 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-12-08T12:40:07.741 INFO:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-08T12:40:07.881 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-08T12:40:17.973 INFO:teuthology.orchestra.run.smithi129.stdout:ntpd: time slew -0.004736s 2019-12-08T12:40:18.011 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit not found. 2019-12-08T12:40:18.094 INFO:teuthology.orchestra.run.smithi129.stdout: remote refid st t when poll reach delay offset jitter 2019-12-08T12:40:18.094 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================== 2019-12-08T12:40:18.094 INFO:teuthology.orchestra.run.smithi129.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:18.094 INFO:teuthology.orchestra.run.smithi129.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:18.094 INFO:teuthology.orchestra.run.smithi129.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:18.095 INFO:teuthology.orchestra.run.smithi129.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:18.095 INFO:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-08T12:40:18.227 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-08T12:40:28.309 INFO:teuthology.orchestra.run.smithi181.stdout:ntpd: time slew -0.003798s 2019-12-08T12:40:28.351 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit not found. 2019-12-08T12:40:28.452 INFO:teuthology.orchestra.run.smithi181.stdout: remote refid st t when poll reach delay offset jitter 2019-12-08T12:40:28.452 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================== 2019-12-08T12:40:28.452 INFO:teuthology.orchestra.run.smithi181.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:28.452 INFO:teuthology.orchestra.run.smithi181.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:28.453 INFO:teuthology.orchestra.run.smithi181.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:28.453 INFO:teuthology.orchestra.run.smithi181.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-08T12:40:28.454 INFO:teuthology.run_tasks:Running task install... 2019-12-08T12:40:28.546 DEBUG:teuthology.task.install:project ceph 2019-12-08T12:40:28.547 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}} 2019-12-08T12:40:28.547 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'} 2019-12-08T12:40:28.547 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-12-08T12:40:28.553 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-12-08T12:40:28.553 INFO:teuthology.task.install:extra packages: [] 2019-12-08T12:40:28.553 INFO:teuthology.orchestra.run.smithi129:> sudo lsb_release -is 2019-12-08T12:40:28.714 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS 2019-12-08T12:40:28.716 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-08T12:40:28.716 INFO:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2019-12-08T12:40:28.716 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-08T12:40:28.717 INFO:teuthology.orchestra.run.smithi129:> sudo lsb_release -is 2019-12-08T12:40:28.850 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS 2019-12-08T12:40:28.853 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-08T12:40:28.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:40:28.886 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS 2019-12-08T12:40:28.887 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-08T12:40:28.887 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-08T12:40:28.887 INFO:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2019-12-08T12:40:29.015 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-08T12:40:29.015 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-08T12:40:29.022 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS 2019-12-08T12:40:29.025 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-08T12:40:29.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:40:29.135 INFO:teuthology.orchestra.run.smithi129:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-08T12:40:29.174 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-08T12:40:29.175 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-08T12:40:29.258 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-08T12:40:29.258 INFO:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-08T12:40:29.292 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-08T12:40:29.372 INFO:teuthology.orchestra.run.smithi129:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-08T12:40:29.414 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-08T12:40:29.414 INFO:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-08T12:40:29.461 INFO:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-08T12:40:29.516 INFO:teuthology.orchestra.run.smithi181:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-08T12:40:29.539 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2019-12-08T12:40:29.541 INFO:teuthology.orchestra.run.smithi129:> sudo yum clean all 2019-12-08T12:40:29.603 INFO:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-08T12:40:29.683 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2019-12-08T12:40:29.685 INFO:teuthology.orchestra.run.smithi181:> sudo yum clean all 2019-12-08T12:40:29.710 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:40:29.749 INFO:teuthology.orchestra.run.smithi129.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-08T12:40:29.749 INFO:teuthology.orchestra.run.smithi129.stdout: : lab-extras updates 2019-12-08T12:40:29.795 INFO:teuthology.orchestra.run.smithi129.stdout:Cleaning up list of fastest mirrors 2019-12-08T12:40:29.796 INFO:teuthology.orchestra.run.smithi129.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-08T12:40:29.808 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw 2019-12-08T12:40:29.860 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:40:29.899 INFO:teuthology.orchestra.run.smithi181.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-08T12:40:29.899 INFO:teuthology.orchestra.run.smithi181.stdout: : lab-extras updates 2019-12-08T12:40:29.945 INFO:teuthology.orchestra.run.smithi181.stdout:Cleaning up list of fastest mirrors 2019-12-08T12:40:29.945 INFO:teuthology.orchestra.run.smithi181.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-08T12:40:29.958 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw 2019-12-08T12:40:29.967 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:40:30.004 INFO:teuthology.orchestra.run.smithi129.stdout:Determining fastest mirrors 2019-12-08T12:40:30.212 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:40:30.257 INFO:teuthology.orchestra.run.smithi181.stdout:Determining fastest mirrors 2019-12-08T12:40:30.649 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:40:30.649 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:40:30.650 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:40:30.650 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:40:30.847 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:40:30.847 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:40:30.848 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:40:30.849 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:40:38.059 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:40:38.416 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:40:38.417 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:40:38.417 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.430 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.537 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.538 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.539 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.541 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.541 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.543 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.543 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.545 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.546 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:40:38.546 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.554 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.556 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.557 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.558 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.559 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.560 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.563 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-08T12:40:38.563 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-08T12:40:38.564 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-08T12:40:38.564 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-08T12:40:38.566 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-08T12:40:38.567 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.568 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:40:38.568 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.572 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.573 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.574 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.575 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.576 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.577 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.578 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.579 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.580 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.580 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.581 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:38.582 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-08T12:40:38.582 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.582 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-08T12:40:38.582 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-08T12:40:38.583 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-08T12:40:38.584 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-08T12:40:38.585 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-08T12:40:38.585 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:40:38.585 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-08T12:40:38.586 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.587 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.587 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.588 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.589 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-08T12:40:38.589 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.590 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.590 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:38.590 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-08T12:40:39.167 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:40:39.455 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:40:39.525 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:40:39.525 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:40:39.526 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.538 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.542 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:40:39.542 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:40:39.545 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-08T12:40:39.546 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-08T12:40:39.547 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-08T12:40:39.548 INFO:teuthology.orchestra.run.smithi129.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout:Updating for dependencies: 2019-12-08T12:40:39.549 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+20 Dependent packages) 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade ( 2 Dependent packages) 2019-12-08T12:40:39.550 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:40:39.551 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 45 M 2019-12-08T12:40:39.551 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:40:39.551 INFO:teuthology.orchestra.run.smithi129.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-08T12:40:39.653 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.653 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.654 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.654 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.654 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.654 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.654 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.655 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.655 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:40:39.655 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.663 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.664 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.665 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.667 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.668 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.668 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.671 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-08T12:40:39.672 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-08T12:40:39.672 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-08T12:40:39.673 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-08T12:40:39.674 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-08T12:40:39.675 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.676 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:40:39.676 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.681 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.682 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.683 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.684 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.684 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.686 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.686 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.687 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.688 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.689 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.689 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:40:39.690 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-08T12:40:39.690 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.691 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-08T12:40:39.691 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-08T12:40:39.691 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-08T12:40:39.692 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-08T12:40:39.693 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-08T12:40:39.693 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:40:39.694 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-08T12:40:39.695 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.695 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.696 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.696 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-08T12:40:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.698 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.698 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:40:39.698 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-08T12:40:40.557 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:40:40.643 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:40:40.644 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:40:40.647 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:40:40.647 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:40:40.647 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:40:40.647 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:40:40.647 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:40:40.647 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:40:40.648 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-08T12:40:40.648 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:40:40.648 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-08T12:40:40.648 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-08T12:40:40.648 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-08T12:40:40.648 INFO:teuthology.orchestra.run.smithi181.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-08T12:40:40.649 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-08T12:40:40.650 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-08T12:40:40.650 INFO:teuthology.orchestra.run.smithi181.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-08T12:40:40.650 INFO:teuthology.orchestra.run.smithi181.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-08T12:40:40.650 INFO:teuthology.orchestra.run.smithi181.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-08T12:40:40.650 INFO:teuthology.orchestra.run.smithi181.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-08T12:40:40.650 INFO:teuthology.orchestra.run.smithi181.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-08T12:40:40.651 INFO:teuthology.orchestra.run.smithi181.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-08T12:40:40.651 INFO:teuthology.orchestra.run.smithi181.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-08T12:40:40.651 INFO:teuthology.orchestra.run.smithi181.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-08T12:40:40.651 INFO:teuthology.orchestra.run.smithi181.stdout:Updating for dependencies: 2019-12-08T12:40:40.651 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-08T12:40:40.651 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+20 Dependent packages) 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade ( 2 Dependent packages) 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:40:40.652 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 45 M 2019-12-08T12:40:40.653 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:40:40.653 INFO:teuthology.orchestra.run.smithi181.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-08T12:40:42.546 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:40:42.546 INFO:teuthology.orchestra.run.smithi129.stdout:Total 15 MB/s | 45 MB 00:02 2019-12-08T12:40:42.564 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:40:42.642 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:40:42.872 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:40:42.873 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:40:43.506 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-08T12:40:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-08T12:40:44.233 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-08T12:40:44.528 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-08T12:40:44.773 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-08T12:40:45.084 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-08T12:40:45.669 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:40:45.669 INFO:teuthology.orchestra.run.smithi181.stdout:Total 8.9 MB/s | 45 MB 00:05 2019-12-08T12:40:45.684 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:40:45.755 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:40:45.963 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:40:45.963 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:40:46.005 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-08T12:40:46.390 INFO:teuthology.orchestra.run.smithi129.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-08T12:40:46.685 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-08T12:40:47.039 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-08T12:40:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-08T12:40:47.413 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-08T12:40:47.632 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-08T12:40:47.741 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-08T12:40:48.049 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-08T12:40:48.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-08T12:40:48.439 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-08T12:40:48.518 INFO:teuthology.orchestra.run.smithi129.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-08T12:40:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-08T12:40:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-08T12:40:49.228 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-08T12:40:49.296 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-08T12:40:49.594 INFO:teuthology.orchestra.run.smithi181.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-08T12:40:49.596 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-08T12:40:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-08T12:40:50.234 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-08T12:40:50.494 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-08T12:40:50.903 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-08T12:40:51.456 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-08T12:40:51.773 INFO:teuthology.orchestra.run.smithi181.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-08T12:40:52.078 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-08T12:40:52.390 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-08T12:40:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-08T12:40:52.742 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-08T12:40:53.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-08T12:40:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-08T12:40:53.690 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-08T12:40:53.735 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-08T12:40:54.257 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-08T12:40:56.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-08T12:40:57.318 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-08T12:40:57.908 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-08T12:41:03.646 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-08T12:41:04.043 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-08T12:41:04.327 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-08T12:41:04.622 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-08T12:41:04.773 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-08T12:41:04.874 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-08T12:41:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-08T12:41:05.125 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-08T12:41:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-08T12:41:06.176 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-08T12:41:06.268 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-08T12:41:06.369 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-08T12:41:06.469 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-08T12:41:06.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-08T12:41:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-08T12:41:06.788 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-08T12:41:06.904 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-08T12:41:07.013 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-08T12:41:07.106 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-08T12:41:07.206 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-08T12:41:07.225 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-08T12:41:07.331 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-08T12:41:07.449 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-08T12:41:07.583 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-08T12:41:07.605 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-08T12:41:07.694 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-08T12:41:07.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-08T12:41:07.884 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-08T12:41:07.898 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-08T12:41:07.976 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-08T12:41:07.978 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-08T12:41:08.317 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-08T12:41:08.317 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.318 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.319 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-08T12:41:08.320 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.320 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-08T12:41:08.320 INFO:teuthology.orchestra.run.smithi129.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.320 INFO:teuthology.orchestra.run.smithi129.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.321 INFO:teuthology.orchestra.run.smithi129.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.321 INFO:teuthology.orchestra.run.smithi129.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-08T12:41:08.321 INFO:teuthology.orchestra.run.smithi129.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.321 INFO:teuthology.orchestra.run.smithi129.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.322 INFO:teuthology.orchestra.run.smithi129.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.322 INFO:teuthology.orchestra.run.smithi129.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-08T12:41:08.323 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:08.323 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Updated: 2019-12-08T12:41:08.323 INFO:teuthology.orchestra.run.smithi129.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.323 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:08.323 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:08.323 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:41:08.441 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-test 2019-12-08T12:41:08.734 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:08.774 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:08.774 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:08.775 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:08.775 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:08.775 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:09.033 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-08T12:41:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-08T12:41:09.278 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-08T12:41:09.404 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-08T12:41:09.426 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:09.504 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-08T12:41:09.638 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-08T12:41:09.757 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:41:09.758 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:09.758 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:09.764 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-08T12:41:09.775 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:09.805 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:09.806 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:09.808 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:09.808 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-08T12:41:09.808 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-08T12:41:09.809 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-08T12:41:09.811 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-08T12:41:09.814 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:09.814 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-08T12:41:09.876 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-08T12:41:10.031 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-08T12:41:10.140 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-08T12:41:10.175 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:10.264 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-08T12:41:10.395 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:10.396 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:41:10.397 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:10.397 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:10.397 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:41:10.397 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-08T12:41:10.398 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+4 Dependent packages) 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:10.399 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 38 M 2019-12-08T12:41:10.400 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 134 M 2019-12-08T12:41:10.400 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:41:10.408 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-08T12:41:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-08T12:41:10.700 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-08T12:41:10.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-08T12:41:10.885 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-08T12:41:10.994 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-08T12:41:11.111 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-08T12:41:11.220 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-08T12:41:11.345 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-08T12:41:11.479 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-08T12:41:11.588 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-08T12:41:11.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-08T12:41:11.847 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-08T12:41:11.849 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-08T12:41:12.185 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:12.186 INFO:teuthology.orchestra.run.smithi129.stdout:Total 21 MB/s | 38 MB 00:01 2019-12-08T12:41:12.188 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:41:12.208 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:41:12.238 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.239 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-08T12:41:12.240 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-08T12:41:12.241 INFO:teuthology.orchestra.run.smithi181.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Updated: 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.242 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:12.243 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:12.243 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:41:12.266 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:41:12.267 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:41:12.359 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-test 2019-12-08T12:41:12.671 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:12.718 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:12.719 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:12.719 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:12.719 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:12.720 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:12.936 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-08T12:41:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-08T12:41:13.373 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:13.534 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-08T12:41:13.715 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:41:13.716 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:13.716 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:13.733 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:13.763 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:13.764 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:13.765 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:13.765 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-08T12:41:13.766 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-08T12:41:13.767 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-08T12:41:13.768 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-08T12:41:13.771 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:13.771 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-08T12:41:14.129 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:14.349 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:14.349 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:41:14.350 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:14.350 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:14.350 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:41:14.350 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:41:14.350 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:14.350 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-08T12:41:14.351 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:14.352 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:41:14.352 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:14.352 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+4 Dependent packages) 2019-12-08T12:41:14.352 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:14.353 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 38 M 2019-12-08T12:41:14.353 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 134 M 2019-12-08T12:41:14.353 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:41:16.195 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:16.195 INFO:teuthology.orchestra.run.smithi181.stdout:Total 21 MB/s | 38 MB 00:01 2019-12-08T12:41:16.197 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:41:16.216 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:41:16.280 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:41:16.281 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:41:17.040 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-08T12:41:17.343 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-08T12:41:17.495 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-08T12:41:17.680 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-08T12:41:17.802 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-08T12:41:17.901 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-08T12:41:17.993 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-08T12:41:18.093 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-08T12:41:18.177 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-08T12:41:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-08T12:41:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-08T12:41:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:18.655 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:41:18.784 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph 2019-12-08T12:41:19.032 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:19.078 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:19.079 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:19.079 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:19.080 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:19.080 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:19.734 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:20.074 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:41:20.075 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.075 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.107 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.108 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.109 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.110 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.110 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:20.120 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:20.122 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.123 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.125 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.126 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.126 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.127 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:20.128 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:20.131 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:20.135 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.135 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-08T12:41:20.136 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-08T12:41:20.137 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-08T12:41:20.138 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:20.139 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:20.139 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:20.140 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:20.141 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:20.142 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-08T12:41:20.142 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-08T12:41:20.143 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-08T12:41:20.144 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-08T12:41:20.144 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-08T12:41:20.145 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.145 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-08T12:41:20.145 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-08T12:41:20.146 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-08T12:41:20.147 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-08T12:41:20.147 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-08T12:41:20.147 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-08T12:41:20.148 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-08T12:41:20.148 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-08T12:41:20.148 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-08T12:41:20.149 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.149 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-08T12:41:20.149 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-08T12:41:20.149 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-08T12:41:20.150 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.150 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-08T12:41:20.150 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-08T12:41:20.150 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:20.151 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-08T12:41:20.513 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:20.737 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:20.737 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:41:20.740 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:20.740 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:20.740 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:41:20.740 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-08T12:41:20.741 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-08T12:41:20.742 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-08T12:41:20.742 INFO:teuthology.orchestra.run.smithi129.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-08T12:41:20.742 INFO:teuthology.orchestra.run.smithi129.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-08T12:41:20.742 INFO:teuthology.orchestra.run.smithi129.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-08T12:41:20.742 INFO:teuthology.orchestra.run.smithi129.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-08T12:41:20.742 INFO:teuthology.orchestra.run.smithi129.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-08T12:41:20.743 INFO:teuthology.orchestra.run.smithi129.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-08T12:41:20.743 INFO:teuthology.orchestra.run.smithi129.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-08T12:41:20.744 INFO:teuthology.orchestra.run.smithi129.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-08T12:41:20.744 INFO:teuthology.orchestra.run.smithi129.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-08T12:41:20.744 INFO:teuthology.orchestra.run.smithi129.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-08T12:41:20.744 INFO:teuthology.orchestra.run.smithi129.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-08T12:41:20.744 INFO:teuthology.orchestra.run.smithi129.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-08T12:41:20.744 INFO:teuthology.orchestra.run.smithi129.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:41:20.745 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:20.746 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+21 Dependent packages) 2019-12-08T12:41:20.746 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:20.746 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 28 M 2019-12-08T12:41:20.747 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 87 M 2019-12-08T12:41:20.747 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:41:21.699 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-08T12:41:22.031 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-08T12:41:22.153 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:22.154 INFO:teuthology.orchestra.run.smithi129.stdout:Total 20 MB/s | 28 MB 00:01 2019-12-08T12:41:22.171 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-08T12:41:22.176 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:41:22.219 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:41:22.272 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-08T12:41:22.397 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-08T12:41:22.490 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:41:22.491 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:41:22.531 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-08T12:41:22.974 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-08T12:41:22.974 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-08T12:41:22.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:22.976 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:41:23.093 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph 2019-12-08T12:41:23.286 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-08T12:41:23.306 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:23.344 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:23.345 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:23.345 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:23.345 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:23.346 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:23.579 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-08T12:41:23.813 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-08T12:41:23.953 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:24.282 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:41:24.283 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.283 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:24.285 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-08T12:41:24.315 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.316 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.317 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.318 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.319 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:24.328 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:24.330 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.331 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.333 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.334 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.335 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.336 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:41:24.337 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:24.339 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:24.344 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.344 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-08T12:41:24.345 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-08T12:41:24.346 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-08T12:41:24.346 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:24.347 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:24.348 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:24.349 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:24.350 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-08T12:41:24.350 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-08T12:41:24.351 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-08T12:41:24.352 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-08T12:41:24.352 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-08T12:41:24.353 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-08T12:41:24.354 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.354 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-08T12:41:24.354 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-08T12:41:24.354 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-08T12:41:24.356 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-08T12:41:24.356 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-08T12:41:24.356 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-08T12:41:24.357 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-08T12:41:24.357 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-08T12:41:24.357 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-08T12:41:24.357 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.357 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-08T12:41:24.358 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-08T12:41:24.359 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-08T12:41:24.359 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.359 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-08T12:41:24.359 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-08T12:41:24.359 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:24.359 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-08T12:41:24.623 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-08T12:41:24.717 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:24.941 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:24.941 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:41:24.944 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:24.944 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:24.944 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:41:24.944 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:41:24.944 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-08T12:41:24.945 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-08T12:41:24.946 INFO:teuthology.orchestra.run.smithi181.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-08T12:41:24.947 INFO:teuthology.orchestra.run.smithi181.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:24.948 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+21 Dependent packages) 2019-12-08T12:41:24.949 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:24.949 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 28 M 2019-12-08T12:41:24.949 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 87 M 2019-12-08T12:41:24.949 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:41:25.028 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-08T12:41:25.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-08T12:41:26.387 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:26.387 INFO:teuthology.orchestra.run.smithi181.stdout:Total 19 MB/s | 28 MB 00:01 2019-12-08T12:41:26.401 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:41:26.428 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:41:26.706 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:41:26.707 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:41:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-08T12:41:27.943 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-08T12:41:28.058 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-08T12:41:28.252 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-08T12:41:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-08T12:41:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-08T12:41:28.820 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-08T12:41:29.146 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-08T12:41:29.632 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-08T12:41:29.900 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-08T12:41:30.096 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-08T12:41:30.315 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-08T12:41:30.623 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-08T12:41:30.625 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-08T12:41:31.193 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-08T12:41:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-08T12:41:31.816 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-08T12:41:32.090 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-08T12:41:32.475 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-08T12:41:32.492 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-08T12:41:32.891 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-08T12:41:33.118 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-08T12:41:33.323 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-08T12:41:33.494 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-08T12:41:33.678 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-08T12:41:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-08T12:41:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-08T12:41:34.011 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-08T12:41:34.136 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-08T12:41:34.261 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-08T12:41:34.353 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-08T12:41:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-08T12:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-08T12:41:34.621 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-08T12:41:34.730 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-08T12:41:34.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-08T12:41:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-08T12:41:34.974 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-08T12:41:34.990 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-08T12:41:35.090 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-08T12:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-08T12:41:35.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-08T12:41:35.322 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-08T12:41:35.416 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-08T12:41:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-08T12:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-08T12:41:35.898 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-08T12:41:36.032 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-08T12:41:36.143 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-08T12:41:36.244 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-08T12:41:36.295 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-08T12:41:36.378 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-08T12:41:36.545 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-08T12:41:36.856 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-08T12:41:36.872 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-08T12:41:36.873 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:36.873 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:41:36.873 INFO:teuthology.orchestra.run.smithi129.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:36.873 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:36.873 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-08T12:41:36.874 INFO:teuthology.orchestra.run.smithi129.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-08T12:41:36.875 INFO:teuthology.orchestra.run.smithi129.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-08T12:41:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:36.877 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:41:37.006 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-daemon 2019-12-08T12:41:37.281 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-08T12:41:37.288 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:37.332 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:37.948 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-08T12:41:37.983 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:38.326 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:41:38.326 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:38.326 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:38.394 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-08T12:41:38.674 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-08T12:41:38.691 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:38.916 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:41:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-08T12:41:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 17 k 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 68 k 2019-12-08T12:41:38.918 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:41:39.018 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-08T12:41:39.160 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-08T12:41:39.234 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:41:39.243 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:41:39.245 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:41:39.245 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:41:39.286 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-08T12:41:39.453 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-08T12:41:39.553 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-08T12:41:39.687 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-08T12:41:39.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-08T12:41:39.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-08T12:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:41:40.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-08T12:41:40.148 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-08T12:41:40.231 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-08T12:41:40.340 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-08T12:41:40.432 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-08T12:41:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:41:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:40.435 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:41:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:40.436 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:40.436 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:41:40.559 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr 2019-12-08T12:41:40.608 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-08T12:41:40.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-08T12:41:40.757 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:40.794 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:40.795 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:40.796 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:40.796 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:40.796 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:40.850 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-08T12:41:40.993 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-08T12:41:41.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-08T12:41:41.277 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-08T12:41:41.394 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-08T12:41:41.431 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:41.532 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-08T12:41:41.770 INFO:teuthology.orchestra.run.smithi129.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:41:41.770 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do 2019-12-08T12:41:41.832 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-dashboard 2019-12-08T12:41:42.049 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:42.089 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:42.090 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-08T12:41:42.090 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:42.090 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:41:42.090 INFO:teuthology.orchestra.run.smithi181.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-08T12:41:42.091 INFO:teuthology.orchestra.run.smithi181.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-08T12:41:42.092 INFO:teuthology.orchestra.run.smithi181.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-08T12:41:42.093 INFO:teuthology.orchestra.run.smithi181.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-08T12:41:42.094 INFO:teuthology.orchestra.run.smithi181.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-08T12:41:42.094 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:42.094 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:41:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:42.223 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-daemon 2019-12-08T12:41:42.456 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:42.495 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:42.495 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:42.496 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:42.496 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:42.497 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-08T12:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-08T12:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-08T12:41:43.144 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:43.466 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:43.474 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:41:43.474 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:43.474 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:43.689 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:43.690 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:41:43.690 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:41:43.691 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+3 Dependent packages) 2019-12-08T12:41:43.692 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:43.693 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 4.6 M 2019-12-08T12:41:43.693 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 20 M 2019-12-08T12:41:43.693 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:41:43.843 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:44.068 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:44.068 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:41:44.069 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:44.069 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:44.069 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:41:44.070 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:44.071 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2019-12-08T12:41:44.071 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:44.071 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 17 k 2019-12-08T12:41:44.071 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 68 k 2019-12-08T12:41:44.071 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:41:44.392 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:44.393 INFO:teuthology.orchestra.run.smithi129.stdout:Total 6.5 MB/s | 4.6 MB 00:00 2019-12-08T12:41:44.398 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:41:44.407 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:41:44.417 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:41:44.425 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:41:44.427 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:41:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:41:44.580 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:41:44.580 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:41:45.221 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-08T12:41:45.306 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:41:45.623 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-08T12:41:45.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:41:45.861 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:45.861 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:41:45.861 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:45.862 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:45.862 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:41:45.989 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr 2019-12-08T12:41:46.255 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:46.298 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:46.299 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:46.299 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:46.299 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:46.300 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:46.952 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:47.291 INFO:teuthology.orchestra.run.smithi181.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:41:47.292 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do 2019-12-08T12:41:47.312 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-08T12:41:47.353 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-dashboard 2019-12-08T12:41:47.595 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:47.643 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:47.644 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:47.644 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:47.644 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:47.645 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-08T12:41:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-08T12:41:47.931 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-08T12:41:48.023 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-08T12:41:48.302 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:48.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-08T12:41:48.500 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:48.500 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:41:48.500 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:48.500 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:48.500 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:41:48.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:48.501 INFO:teuthology.orchestra.run.smithi129.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-08T12:41:48.501 INFO:teuthology.orchestra.run.smithi129.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-08T12:41:48.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:48.501 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:41:48.632 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-08T12:41:48.649 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:41:48.650 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:48.650 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:48.653 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:41:48.683 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:41:48.684 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:48.684 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:48.684 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-08T12:41:48.685 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-08T12:41:48.685 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:48.685 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-08T12:41:48.870 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:48.912 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:48.913 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:48.914 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:49.052 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:49.277 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:49.277 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:41:49.278 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:49.278 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:49.278 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:41:49.278 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-08T12:41:49.279 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:49.280 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:41:49.280 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:49.280 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+3 Dependent packages) 2019-12-08T12:41:49.280 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:49.280 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 4.6 M 2019-12-08T12:41:49.280 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 20 M 2019-12-08T12:41:49.281 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:41:49.565 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:49.901 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:41:49.901 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:49.901 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:50.225 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:50.226 INFO:teuthology.orchestra.run.smithi181.stdout:Total 4.8 MB/s | 4.6 MB 00:00 2019-12-08T12:41:50.232 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:41:50.241 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:41:50.266 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:50.415 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:41:50.416 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:41:50.491 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:50.491 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:41:50.492 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:50.492 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:50.493 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:41:50.493 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:50.493 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:41:50.493 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-cloud 2019-12-08T12:41:50.493 INFO:teuthology.orchestra.run.smithi129.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-08T12:41:50.493 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 82 k 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 411 k 2019-12-08T12:41:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:41:50.898 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:41:50.902 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:41:51.065 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:41:51.065 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:41:51.114 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-08T12:41:51.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-08T12:41:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-08T12:41:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-08T12:41:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:52.664 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:41:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:52.665 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:41:52.720 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-08T12:41:52.790 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-08T12:41:53.046 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:53.091 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:53.181 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-08T12:41:53.323 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-08T12:41:53.515 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-08T12:41:53.641 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-08T12:41:53.770 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:54.112 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:41:54.112 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:54.112 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:54.113 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:41:54.144 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:54.144 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-08T12:41:54.154 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-08T12:41:54.154 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-08T12:41:54.155 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-08T12:41:54.155 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:54.155 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-08T12:41:54.155 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-08T12:41:54.156 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-08T12:41:54.157 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-08T12:41:54.157 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:41:54.158 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-08T12:41:54.521 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:54.746 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:54.746 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:41:54.747 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:54.747 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:54.747 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:41:54.747 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:54.747 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:41:54.747 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-08T12:41:54.748 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:54.749 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:41:54.749 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:41:54.749 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+4 Dependent packages) 2019-12-08T12:41:54.749 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:41:54.750 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 19 M 2019-12-08T12:41:54.750 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 104 M 2019-12-08T12:41:54.750 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:41:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-08T12:41:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:54.935 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:41:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:54.936 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:41:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-08T12:41:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-08T12:41:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:54.936 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:41:55.066 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-08T12:41:55.340 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:55.387 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:55.387 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:55.388 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:55.388 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:55.488 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:41:55.488 INFO:teuthology.orchestra.run.smithi129.stdout:Total 26 MB/s | 19 MB 00:00 2019-12-08T12:41:55.500 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:41:55.530 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:41:55.686 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:41:55.687 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:41:56.048 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:56.385 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:41:56.386 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:56.386 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:56.673 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-08T12:41:56.748 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:41:56.973 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:56.973 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:41:56.974 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:56.974 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:56.974 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:41:56.974 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:56.975 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:41:56.975 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-cloud 2019-12-08T12:41:56.975 INFO:teuthology.orchestra.run.smithi181.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-08T12:41:56.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:56.975 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:41:56.975 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:41:56.976 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2019-12-08T12:41:56.976 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:56.976 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 82 k 2019-12-08T12:41:56.976 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 411 k 2019-12-08T12:41:56.976 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:41:57.172 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-08T12:41:57.280 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:41:57.283 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:41:57.337 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:41:57.338 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:41:58.074 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-08T12:41:58.529 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-08T12:41:58.529 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:58.529 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:41:58.530 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:41:58.530 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:41:58.530 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:41:58.651 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-08T12:41:58.910 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:41:58.932 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-08T12:41:58.960 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:41:58.961 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:41:58.961 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:41:58.961 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:41:58.961 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:41:59.605 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:41:59.949 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:41:59.949 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:59.950 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:41:59.952 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:41:59.984 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:59.984 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-08T12:41:59.989 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-08T12:41:59.990 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-08T12:41:59.992 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-08T12:41:59.992 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:59.993 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-08T12:41:59.993 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-08T12:41:59.994 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-08T12:41:59.996 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-08T12:41:59.996 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:41:59.996 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-08T12:42:00.357 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:00.581 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:00.581 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:00.582 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:00.582 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:00.582 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:42:00.582 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:00.582 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:00.582 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-08T12:42:00.583 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+4 Dependent packages) 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 19 M 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 104 M 2019-12-08T12:42:00.584 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:00.614 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-08T12:42:00.898 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-08T12:42:01.003 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-08T12:42:01.248 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:01.248 INFO:teuthology.orchestra.run.smithi181.stdout:Total 29 MB/s | 19 MB 00:00 2019-12-08T12:42:01.258 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:01.285 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:01.412 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:01.413 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:01.714 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-08T12:42:01.829 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-08T12:42:01.930 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-08T12:42:02.398 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-08T12:42:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-08T12:42:02.423 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:02.423 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:02.424 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:02.554 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-rook 2019-12-08T12:42:02.828 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:02.875 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:02.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-08T12:42:03.530 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:03.871 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:42:03.871 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:03.872 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:03.873 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:42:03.904 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:03.904 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-08T12:42:03.906 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:03.907 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:03.908 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:03.909 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:03.910 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:03.910 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-08T12:42:03.910 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-08T12:42:03.911 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-08T12:42:03.911 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-08T12:42:03.912 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-08T12:42:03.913 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-08T12:42:03.914 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-08T12:42:03.914 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-08T12:42:03.915 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:03.915 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-08T12:42:03.915 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-08T12:42:03.916 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-08T12:42:03.917 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-08T12:42:03.917 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-08T12:42:03.917 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:03.917 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-08T12:42:04.287 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:04.508 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:42:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:04.509 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:42:04.509 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-08T12:42:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-08T12:42:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:04.512 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:42:04.512 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:04.512 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+10 Dependent packages) 2019-12-08T12:42:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:04.513 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 1.6 M 2019-12-08T12:42:04.513 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 24 M 2019-12-08T12:42:04.513 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:42:04.636 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-08T12:42:05.327 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:05.328 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.0 MB/s | 1.6 MB 00:00 2019-12-08T12:42:05.342 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:42:05.354 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:42:05.422 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:42:05.423 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:42:06.303 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-08T12:42:06.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-08T12:42:06.637 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-08T12:42:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-08T12:42:06.737 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-08T12:42:06.845 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-08T12:42:06.988 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-08T12:42:07.050 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-08T12:42:07.096 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-08T12:42:07.330 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-08T12:42:07.573 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-08T12:42:07.573 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:07.573 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:07.574 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:07.574 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:07.574 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:42:07.574 INFO:teuthology.orchestra.run.smithi181.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-08T12:42:07.574 INFO:teuthology.orchestra.run.smithi181.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-08T12:42:07.574 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:07.575 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-08T12:42:07.707 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-rook 2019-12-08T12:42:07.929 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:07.970 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:07.971 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:07.972 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:07.972 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:07.972 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:08.003 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-08T12:42:08.334 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-08T12:42:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-08T12:42:08.621 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:08.954 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:42:08.955 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:08.955 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:08.957 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:42:08.987 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:08.988 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-08T12:42:08.989 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:08.991 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:08.991 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:08.992 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-08T12:42:08.993 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:08.994 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-08T12:42:08.994 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-08T12:42:08.994 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-08T12:42:08.995 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-08T12:42:09.000 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-08T12:42:09.000 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-08T12:42:09.000 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-08T12:42:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-08T12:42:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-08T12:42:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-08T12:42:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-08T12:42:09.001 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-08T12:42:09.002 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-08T12:42:09.002 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:09.002 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-08T12:42:09.363 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:09.590 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:09.592 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:09.592 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:09.592 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:42:09.592 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:09.592 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:09.593 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-08T12:42:09.593 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:42:09.593 INFO:teuthology.orchestra.run.smithi181.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-08T12:42:09.593 INFO:teuthology.orchestra.run.smithi181.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-08T12:42:09.593 INFO:teuthology.orchestra.run.smithi181.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-08T12:42:09.593 INFO:teuthology.orchestra.run.smithi181.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-08T12:42:09.594 INFO:teuthology.orchestra.run.smithi181.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-08T12:42:09.594 INFO:teuthology.orchestra.run.smithi181.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-08T12:42:09.594 INFO:teuthology.orchestra.run.smithi181.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-08T12:42:09.594 INFO:teuthology.orchestra.run.smithi181.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-08T12:42:09.594 INFO:teuthology.orchestra.run.smithi181.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+10 Dependent packages) 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:09.595 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 1.6 M 2019-12-08T12:42:09.596 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 24 M 2019-12-08T12:42:09.596 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:09.928 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-08T12:42:10.281 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-08T12:42:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.9 MB/s | 1.6 MB 00:00 2019-12-08T12:42:10.456 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:10.543 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:10.543 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:10.551 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-08T12:42:10.691 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-08T12:42:10.808 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-08T12:42:10.901 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-08T12:42:10.968 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-08T12:42:11.051 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-08T12:42:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-08T12:42:11.291 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-08T12:42:11.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-08T12:42:11.428 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-08T12:42:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-08T12:42:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-08T12:42:11.678 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-08T12:42:12.059 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-08T12:42:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-08T12:42:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:12.173 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-08T12:42:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-08T12:42:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:12.176 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:12.302 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-ssh 2019-12-08T12:42:12.380 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-08T12:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:12.608 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:12.609 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:12.609 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:12.610 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:12.610 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:12.755 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-08T12:42:13.157 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-08T12:42:13.260 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:13.512 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-08T12:42:13.602 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:42:13.603 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:13.603 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:13.605 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:42:13.637 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:13.637 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-08T12:42:13.637 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-08T12:42:13.638 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:13.638 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-08T12:42:13.814 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-08T12:42:14.003 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:14.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:14.231 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:42:14.232 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:14.232 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:14.233 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:42:14.233 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:14.233 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:42:14.233 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-08T12:42:14.234 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:42:14.234 INFO:teuthology.orchestra.run.smithi129.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-08T12:42:14.234 INFO:teuthology.orchestra.run.smithi129.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-08T12:42:14.234 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:14.235 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:42:14.235 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:14.235 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+2 Dependent packages) 2019-12-08T12:42:14.235 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:14.236 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 316 k 2019-12-08T12:42:14.236 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.1 M 2019-12-08T12:42:14.236 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:42:14.766 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:14.766 INFO:teuthology.orchestra.run.smithi129.stdout:Total 592 kB/s | 316 kB 00:00 2019-12-08T12:42:14.770 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:42:14.779 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:42:14.858 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:42:14.858 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:42:15.234 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-08T12:42:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-08T12:42:15.700 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-08T12:42:15.918 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-08T12:42:15.939 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-08T12:42:16.033 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-08T12:42:16.192 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-08T12:42:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-08T12:42:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-08T12:42:16.368 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-08T12:42:16.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-08T12:42:16.485 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-08T12:42:16.520 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-08T12:42:16.594 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-08T12:42:16.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-08T12:42:16.870 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-08T12:42:16.971 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-08T12:42:16.998 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-08T12:42:16.998 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:16.998 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:16.998 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:16.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:16.999 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:42:16.999 INFO:teuthology.orchestra.run.smithi129.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-08T12:42:16.999 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:17.000 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:17.071 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-08T12:42:17.134 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-fuse 2019-12-08T12:42:17.352 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:17.390 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:17.392 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:17.393 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:17.623 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-08T12:42:17.623 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:17.624 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:17.624 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:17.624 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:17.624 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:42:17.624 INFO:teuthology.orchestra.run.smithi181.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-08T12:42:17.624 INFO:teuthology.orchestra.run.smithi181.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-08T12:42:17.625 INFO:teuthology.orchestra.run.smithi181.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-08T12:42:17.626 INFO:teuthology.orchestra.run.smithi181.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-08T12:42:17.626 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:17.626 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:17.755 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-ssh 2019-12-08T12:42:17.940 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:17.977 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:17.978 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:17.981 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:17.981 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:17.981 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:18.042 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:18.370 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:42:18.371 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:18.371 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:18.381 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:42:18.414 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:42:18.415 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:18.415 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-08T12:42:18.415 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:18.417 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:18.417 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:18.419 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:18.419 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:18.419 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-08T12:42:18.425 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-08T12:42:18.425 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-08T12:42:18.586 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:18.788 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:18.916 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:42:18.916 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:18.916 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:18.918 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-08T12:42:18.948 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-08T12:42:18.949 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-08T12:42:18.950 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:18.950 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-08T12:42:19.013 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:19.013 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:42:19.014 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:19.015 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:19.015 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:42:19.015 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:42:19.015 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:19.015 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:42:19.015 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-08T12:42:19.016 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:42:19.016 INFO:teuthology.orchestra.run.smithi129.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-08T12:42:19.016 INFO:teuthology.orchestra.run.smithi129.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-08T12:42:19.016 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-08T12:42:19.016 INFO:teuthology.orchestra.run.smithi129.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-08T12:42:19.016 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:19.017 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:42:19.017 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:19.017 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+4 Dependent packages) 2019-12-08T12:42:19.017 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:19.017 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 10 M 2019-12-08T12:42:19.017 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 49 M 2019-12-08T12:42:19.018 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:42:19.316 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:19.527 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:19.527 INFO:teuthology.orchestra.run.smithi129.stdout:Total 19 MB/s | 10 MB 00:00 2019-12-08T12:42:19.541 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:42:19.544 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:19.544 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:19.545 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:42:19.546 INFO:teuthology.orchestra.run.smithi181.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+2 Dependent packages) 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:19.547 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 316 k 2019-12-08T12:42:19.548 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 1.1 M 2019-12-08T12:42:19.548 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:19.555 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:42:19.665 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:42:19.665 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:42:20.079 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:20.080 INFO:teuthology.orchestra.run.smithi181.stdout:Total 592 kB/s | 316 kB 00:00 2019-12-08T12:42:20.083 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:20.092 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:20.175 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:20.175 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:21.050 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-08T12:42:21.389 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-08T12:42:21.708 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-08T12:42:21.828 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-08T12:42:21.954 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-08T12:42:22.022 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-08T12:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-08T12:42:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-08T12:42:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:22.497 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:22.498 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:42:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-08T12:42:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:22.498 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:22.631 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-fuse 2019-12-08T12:42:22.850 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:22.895 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:23.367 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-08T12:42:23.537 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:23.839 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-08T12:42:23.866 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:42:23.867 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:23.867 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:23.877 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:42:23.911 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:42:23.911 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:23.911 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-08T12:42:23.912 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:23.913 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:23.914 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:23.915 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-08T12:42:23.915 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:23.915 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-08T12:42:23.921 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-08T12:42:23.921 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-08T12:42:24.124 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-08T12:42:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-08T12:42:24.284 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:24.332 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-08T12:42:24.424 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-08T12:42:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:24.509 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:24.510 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:42:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:42:24.510 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:24.510 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-08T12:42:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:24.512 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:24.512 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:24.512 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+4 Dependent packages) 2019-12-08T12:42:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:24.512 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 10 M 2019-12-08T12:42:24.513 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 49 M 2019-12-08T12:42:24.513 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:24.584 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-08T12:42:24.932 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:24.933 INFO:teuthology.orchestra.run.smithi181.stdout:Total 24 MB/s | 10 MB 00:00 2019-12-08T12:42:24.942 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:24.957 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:25.017 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-08T12:42:25.018 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-08T12:42:25.019 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:25.019 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:25.033 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:25.034 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:25.146 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs2 2019-12-08T12:42:25.380 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:25.420 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:26.072 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:26.402 INFO:teuthology.orchestra.run.smithi129.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:42:26.402 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do 2019-12-08T12:42:26.464 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs-devel 2019-12-08T12:42:26.722 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:26.768 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:26.769 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:26.770 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:26.770 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:26.771 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:27.355 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-08T12:42:27.417 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:27.759 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:42:27.760 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:27.760 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:27.761 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:42:27.791 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:27.791 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:27.892 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-08T12:42:28.163 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:28.389 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:28.390 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:42:28.390 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-08T12:42:28.391 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:42:28.392 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-08T12:42:28.392 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:28.392 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:42:28.392 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:28.392 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package (+1 Dependent package) 2019-12-08T12:42:28.392 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:28.393 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 170 k 2019-12-08T12:42:28.393 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 679 k 2019-12-08T12:42:28.393 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:42:28.773 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:28.773 INFO:teuthology.orchestra.run.smithi129.stdout:Total 445 kB/s | 170 kB 00:00 2019-12-08T12:42:28.776 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:42:28.786 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:42:28.789 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:42:28.789 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:42:28.822 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-08T12:42:29.332 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-08T12:42:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-08T12:42:29.607 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-08T12:42:29.658 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-08T12:42:29.724 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-08T12:42:29.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-08T12:42:29.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-08T12:42:29.967 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-08T12:42:30.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-08T12:42:30.214 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-08T12:42:30.214 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:30.215 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:30.346 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install librados2 2019-12-08T12:42:30.604 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:30.618 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-08T12:42:30.618 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-08T12:42:30.619 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:30.620 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:30.649 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:30.650 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:30.651 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:30.651 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:30.651 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:30.748 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs2 2019-12-08T12:42:30.984 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:31.026 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:31.027 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:31.027 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:31.027 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:31.028 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:31.297 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:31.647 INFO:teuthology.orchestra.run.smithi129.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:42:31.647 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do 2019-12-08T12:42:31.678 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:31.713 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install librbd1 2019-12-08T12:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:32.015 INFO:teuthology.orchestra.run.smithi181.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:42:32.015 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do 2019-12-08T12:42:32.077 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs-devel 2019-12-08T12:42:32.233 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:32.267 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:32.269 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:32.637 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:32.855 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:32.972 INFO:teuthology.orchestra.run.smithi129.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:42:32.972 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do 2019-12-08T12:42:33.034 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install python-ceph 2019-12-08T12:42:33.182 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:42:33.183 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:33.183 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:33.184 INFO:teuthology.orchestra.run.smithi181.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-08T12:42:33.214 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:33.215 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:33.253 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:33.293 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:33.586 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:33.813 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:33.813 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:33.814 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:33.814 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:33.814 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:42:33.814 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:42:33.814 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:33.814 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:33.815 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-08T12:42:33.815 INFO:teuthology.orchestra.run.smithi181.stdout:Installing for dependencies: 2019-12-08T12:42:33.815 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-08T12:42:33.815 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:33.815 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:33.816 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:33.816 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package (+1 Dependent package) 2019-12-08T12:42:33.816 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:33.816 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 170 k 2019-12-08T12:42:33.816 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 679 k 2019-12-08T12:42:33.816 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:33.942 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:34.197 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:42:34.197 INFO:teuthology.orchestra.run.smithi181.stdout:Total 443 kB/s | 170 kB 00:00 2019-12-08T12:42:34.200 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:34.210 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:34.212 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:34.213 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:34.277 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:42:34.278 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:34.278 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:34.644 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:34.830 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:34.830 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:42:34.831 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:34.831 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:34.831 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository 2019-12-08T12:42:34.831 INFO:teuthology.orchestra.run.smithi129.stdout: Size 2019-12-08T12:42:34.831 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2019-12-08T12:42:34.832 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:34.833 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 2.2 k 2019-12-08T12:42:34.833 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2019-12-08T12:42:34.833 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:42:35.048 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-08T12:42:35.127 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:42:35.129 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:42:35.131 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:42:35.131 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:42:35.301 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-08T12:42:35.458 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-08T12:42:35.744 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:35.943 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-08T12:42:35.943 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:35.943 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:35.943 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:35.943 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:35.944 INFO:teuthology.orchestra.run.smithi181.stdout:Dependency Installed: 2019-12-08T12:42:35.944 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:35.944 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:35.944 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:36.077 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install librados2 2019-12-08T12:42:36.253 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:36.256 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:36.256 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:36.256 INFO:teuthology.orchestra.run.smithi129.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:36.257 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:36.257 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:36.308 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:36.349 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:36.349 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:36.350 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:36.350 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:36.350 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:36.366 INFO:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-fuse 2019-12-08T12:42:36.614 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:36.659 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:36.660 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:36.661 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:36.661 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:36.661 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:36.992 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:37.316 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:37.322 INFO:teuthology.orchestra.run.smithi181.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:42:37.323 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do 2019-12-08T12:42:37.384 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install librbd1 2019-12-08T12:42:37.573 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:37.608 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:37.608 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:37.609 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:37.657 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:42:37.658 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:42:37.658 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:38.036 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:38.198 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:38.220 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:38.220 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:42:38.221 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:38.221 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:38.221 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:42:38.221 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2019-12-08T12:42:38.222 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:38.223 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 76 k 2019-12-08T12:42:38.223 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 200 k 2019-12-08T12:42:38.223 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:42:38.525 INFO:teuthology.orchestra.run.smithi181.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-08T12:42:38.526 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do 2019-12-08T12:42:38.587 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install python-ceph 2019-12-08T12:42:38.754 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:42:38.773 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:42:38.774 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:42:38.775 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:42:38.796 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:38.839 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:38.840 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:38.841 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:38.841 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:38.841 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:39.492 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:39.768 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:42:39.768 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:39.769 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:42:39.769 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:42:39.829 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:42:39.829 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:39.830 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:39.880 DEBUG:teuthology.parallel:result is None 2019-12-08T12:42:40.198 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:40.387 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:40.387 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout: Size 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:40.388 INFO:teuthology.orchestra.run.smithi181.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 2.2 k 2019-12-08T12:42:40.389 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2019-12-08T12:42:40.390 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:40.684 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:40.685 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:40.687 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:40.688 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:41.328 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:41.787 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:41.788 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:41.788 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:41.788 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:41.902 INFO:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-fuse 2019-12-08T12:42:42.163 INFO:teuthology.orchestra.run.smithi181.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:42:42.208 INFO:teuthology.orchestra.run.smithi181.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:42:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: * base: mirrors.umflint.edu 2019-12-08T12:42:42.209 INFO:teuthology.orchestra.run.smithi181.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:42:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: * extras: mirror.facebook.net 2019-12-08T12:42:42.210 INFO:teuthology.orchestra.run.smithi181.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:42:42.859 INFO:teuthology.orchestra.run.smithi181.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:42:43.198 INFO:teuthology.orchestra.run.smithi181.stdout:Resolving Dependencies 2019-12-08T12:42:43.199 INFO:teuthology.orchestra.run.smithi181.stdout:--> Running transaction check 2019-12-08T12:42:43.199 INFO:teuthology.orchestra.run.smithi181.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-08T12:42:43.574 INFO:teuthology.orchestra.run.smithi181.stdout:--> Finished Dependency Resolution 2019-12-08T12:42:43.760 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:43.760 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies Resolved 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-08T12:42:43.761 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 76 k 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 200 k 2019-12-08T12:42:43.762 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading packages: 2019-12-08T12:42:44.062 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2019-12-08T12:42:44.081 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2019-12-08T12:42:44.083 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded 2019-12-08T12:42:44.083 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2019-12-08T12:42:44.842 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:45.326 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-08T12:42:45.326 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:45.326 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2019-12-08T12:42:45.326 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:45.327 INFO:teuthology.orchestra.run.smithi181.stdout: 2019-12-08T12:42:45.327 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2019-12-08T12:42:45.437 DEBUG:teuthology.parallel:result is None 2019-12-08T12:42:45.438 INFO:teuthology.orchestra.run.smithi129:> sudo lsb_release -is 2019-12-08T12:42:45.595 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS 2019-12-08T12:42:45.597 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-08T12:42:45.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:42:45.761 INFO:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-08T12:42:45.899 INFO:teuthology.orchestra.run.smithi129.stdout:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:45.899 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-08T12:42:45.899 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-08T12:42:45.900 INFO:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2019-12-08T12:42:46.050 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS 2019-12-08T12:42:46.052 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-08T12:42:46.052 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:42:46.242 INFO:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-08T12:42:46.368 INFO:teuthology.orchestra.run.smithi181.stdout:15.0.0-7900.g1351c81.el7 2019-12-08T12:42:46.369 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-08T12:42:46.369 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-08T12:42:46.369 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-12-08T12:42:46.369 INFO:teuthology.orchestra.run.smithi129:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-08T12:42:46.489 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-08T12:42:46.605 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-12-08T12:42:46.605 INFO:teuthology.orchestra.run.smithi129:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-08T12:42:46.726 INFO:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-08T12:42:46.831 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-08T12:42:46.951 INFO:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-08T12:42:47.051 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-12-08T12:42:47.052 INFO:teuthology.orchestra.run.smithi129:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-08T12:42:47.171 INFO:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-08T12:42:47.277 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-08T12:42:47.396 INFO:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-08T12:42:47.508 INFO:teuthology.run_tasks:Running task ceph... 2019-12-08T12:42:47.588 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-12-08T12:42:47.588 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2019-12-08T12:42:47.593 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2019-12-08T12:42:47.708 INFO:tasks.ceph:Disabling ceph logrotate... 2019-12-08T12:42:47.708 INFO:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-08T12:42:47.713 INFO:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-08T12:42:47.823 INFO:tasks.ceph:Creating extra log directories... 2019-12-08T12:42:47.823 INFO:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-08T12:42:47.827 INFO:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-08T12:42:47.931 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-12-08T12:42:47.931 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 1, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'osd shutdown pgref assert': True, 'debug journal': 20, 'bluestore allocator': 'bitmap', 'mon osd nearfull ratio': 0.8, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'osd fast shutdown': False, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-12-08T12:42:47.932 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.22739', 'owner': 'scheduled_ofriedma@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi129.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK8tUMz9/A8JLSzurpLawRtI4Ag2KYFCFJYF1EhE0Dc6hrGCmge1QchfjO/h3Lu6ME/AuCB/Yqlh6DaPVTvX2SDYeHIlIjIquf5iSPU5CcU5+M7S5VNPnDPvMkb/IB08iQHEahu9j/bXyP9dms696Y83TIiOJMEKhrfdZBlDXUqE+2ixtbCL7a3C7qLRb9F6s/Ce/Jp0za1Q8e45sc05ypv2fQ4Den/OQdvrCpWkwbBFCBRXR8FPm80M3HKv02jEMbBNcsi7NI5OrTPdsn4b7nUyO/Jyb3ZHeoYxTOVbigs6GplcDjM8ZXosjor4IB+SWqCYVGxKxYuuSHAKxoK/6X', 'smithi181.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCusmxpv7n8K5eMtzAHkq/cBvkn+DgwKs+Dg/Vm7O1R9EsgZPA18a/b9mkD37Y6pYpcM5o6pO1ulGPGKeiFtaZMlUyH4Nt5ywhEd6jsUg2gQMK2Lc0/L/CVwi7SeCuwOvWHRag9jzHl2mhfeVbWxGcniIQB6pCZ/tJ7QT1YM2j0IwOlDc3taQNDgxOc7oEdUAklKKva/O+huCvgygxHdcFKrke21d5t5Hq7sT+kuBtogao/QuC/wy5kzFbe0Z+NSE12l7k+NcoTyNp9aFJbvxTiFf0lVBHr+jVfoeofYTOfh6GjGYaFXfr8a4B8xwiqF9KNGjX8YtLfgEdLZwiwijwV'}, 'job_id': '4581813', 'tube': 'smithi', 'suite_branch': 'wip-ofriedma-dynamic-disable', 'priority': 30, 'first_in_suite': False, 'branch': 'wip-ofriedma-dynamic-disable', 'suite': 'rgw', 'email': 'ofriedma@redhat.com', 'archive_path': '/home/teuthworker/archive/ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4581813', 'description': 'rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'name': 'ofriedma-2019-12-08_11:30:09-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'os_type': 'centos', 'overrides': {'s3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'mon osd full ratio': 0.9, 'mon osd backfillfull_ratio': 0.85, 'bluestore fsck on mount': True, 'mon osd nearfull ratio': 0.8, 'debug bluestore': 20, 'debug bluefs': 20, 'osd objectstore': 'bluestore', 'bluestore block size': 96636764160, 'debug rocksdb': 10, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95, 'bdev async discard': True}}}, 'rgw': {'frontend': 'beast', 'ec-data-pool': True, 'client.1': {'ssl certificate': 'rgw.client.1'}, 'client.0': {'ssl certificate': 'rgw.client.0'}, 'storage classes': 'LUKEWARM, FROZEN', 'compression type': 'random'}, 'workunit': {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'branch': 'wip-ofriedma-dynamic-disable'}, 'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 1, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'osd shutdown pgref assert': True, 'debug journal': 20, 'bluestore allocator': 'bitmap', 'mon osd nearfull ratio': 0.8, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'osd fast shutdown': False, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}, 'openssl_keys': {'rgw.client.1': {'ca': 'root', 'client': 'client.1', 'embed-key': True}, 'root': {'key-type': 'rsa:4096', 'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0', 'client.1']}, 'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, 'admin_socket': {'branch': 'wip-ofriedma-dynamic-disable'}, 'thrashosds': {'bdev_inject_crash_probability': 0.5, 'bdev_inject_crash': 2}}} 2019-12-08T12:42:47.932 INFO:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-08T12:42:47.936 INFO:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-08T12:42:48.009 INFO:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2019-12-08T12:42:48.013 INFO:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2019-12-08T12:42:48.179 DEBUG:teuthology.orchestra.remote:smithi129:/scratch_devs is 72B 2019-12-08T12:42:48.247 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-08T12:42:48.247 INFO:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-08T12:42:48.325 INFO:teuthology.orchestra.run.smithi129.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-08T12:42:48.326 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:48.326 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 106763 Links: 1 2019-12-08T12:42:48.326 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:48.326 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:48.326 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2019-12-08 12:41:33.397352155 +0000 2019-12-08T12:42:48.327 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2019-12-08 12:39:48.324298746 +0000 2019-12-08T12:42:48.327 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2019-12-08 12:39:48.324298746 +0000 2019-12-08T12:42:48.327 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2019-12-08T12:42:48.352 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2019-12-08T12:42:48.352 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2019-12-08T12:42:48.352 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes (512 B) copied, 0.000432792 s, 1.2 MB/s 2019-12-08T12:42:48.356 INFO:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-08T12:42:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-08T12:42:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:48.420 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 106799 Links: 1 2019-12-08T12:42:48.420 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:48.421 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:48.421 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2019-12-08 12:41:33.397352155 +0000 2019-12-08T12:42:48.421 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2019-12-08 12:39:48.805287038 +0000 2019-12-08T12:42:48.421 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2019-12-08 12:39:48.805287038 +0000 2019-12-08T12:42:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2019-12-08T12:42:48.439 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2019-12-08T12:42:48.439 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2019-12-08T12:42:48.440 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes (512 B) copied, 0.000350559 s, 1.5 MB/s 2019-12-08T12:42:48.444 INFO:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-08T12:42:48.496 INFO:teuthology.orchestra.run.smithi129.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-08T12:42:48.497 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:48.497 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 106257 Links: 1 2019-12-08T12:42:48.497 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:48.497 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:48.497 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2019-12-08 12:41:33.397352155 +0000 2019-12-08T12:42:48.498 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2019-12-08 12:39:49.260275964 +0000 2019-12-08T12:42:48.498 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2019-12-08 12:39:49.260275964 +0000 2019-12-08T12:42:48.498 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2019-12-08T12:42:48.514 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2019-12-08T12:42:48.514 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2019-12-08T12:42:48.515 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes (512 B) copied, 0.00047371 s, 1.1 MB/s 2019-12-08T12:42:48.518 INFO:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-08T12:42:48.573 INFO:teuthology.orchestra.run.smithi129.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-08T12:42:48.574 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:48.574 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 106291 Links: 1 2019-12-08T12:42:48.574 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:48.574 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:48.574 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2019-12-08 12:41:33.397352155 +0000 2019-12-08T12:42:48.574 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2019-12-08 12:39:49.746264135 +0000 2019-12-08T12:42:48.575 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2019-12-08 12:39:49.746264135 +0000 2019-12-08T12:42:48.575 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2019-12-08T12:42:48.593 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2019-12-08T12:42:48.593 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2019-12-08T12:42:48.593 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes (512 B) copied, 0.000335349 s, 1.5 MB/s 2019-12-08T12:42:48.597 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-08T12:42:48.597 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-08T12:42:48.597 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-08T12:42:48.597 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-08T12:42:48.690 DEBUG:teuthology.orchestra.remote:smithi181:/scratch_devs is 72B 2019-12-08T12:42:48.779 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-08T12:42:48.779 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-08T12:42:48.854 INFO:teuthology.orchestra.run.smithi181.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-08T12:42:48.854 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:48.854 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 103828 Links: 1 2019-12-08T12:42:48.854 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:48.854 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:48.855 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2019-12-08 12:41:38.246995816 +0000 2019-12-08T12:42:48.855 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2019-12-08 12:39:48.584771123 +0000 2019-12-08T12:42:48.855 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2019-12-08 12:39:48.584771123 +0000 2019-12-08T12:42:48.855 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2019-12-08T12:42:48.884 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2019-12-08T12:42:48.884 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2019-12-08T12:42:48.884 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes (512 B) copied, 0.000463693 s, 1.1 MB/s 2019-12-08T12:42:48.890 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-08T12:42:48.958 INFO:teuthology.orchestra.run.smithi181.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-08T12:42:48.958 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:48.959 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 108540 Links: 1 2019-12-08T12:42:48.959 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:48.959 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:48.959 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2019-12-08 12:41:38.246995816 +0000 2019-12-08T12:42:48.959 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2019-12-08 12:39:49.073761145 +0000 2019-12-08T12:42:48.960 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2019-12-08 12:39:49.073761145 +0000 2019-12-08T12:42:48.960 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2019-12-08T12:42:48.981 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2019-12-08T12:42:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2019-12-08T12:42:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes (512 B) copied, 0.000432997 s, 1.2 MB/s 2019-12-08T12:42:48.985 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-08T12:42:49.052 INFO:teuthology.orchestra.run.smithi181.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-08T12:42:49.052 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:49.052 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 109617 Links: 1 2019-12-08T12:42:49.052 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:49.052 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:49.053 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2019-12-08 12:41:38.246995816 +0000 2019-12-08T12:42:49.053 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2019-12-08 12:39:49.542751573 +0000 2019-12-08T12:42:49.053 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2019-12-08 12:39:49.542751573 +0000 2019-12-08T12:42:49.053 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2019-12-08T12:42:49.074 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2019-12-08T12:42:49.076 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2019-12-08T12:42:49.077 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes (512 B) copied, 0.000328104 s, 1.6 MB/s 2019-12-08T12:42:49.078 INFO:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-08T12:42:49.134 INFO:teuthology.orchestra.run.smithi181.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-08T12:42:49.134 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-08T12:42:49.134 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 105876 Links: 1 2019-12-08T12:42:49.134 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-08T12:42:49.135 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2019-12-08T12:42:49.135 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2019-12-08 12:41:38.247995915 +0000 2019-12-08T12:42:49.135 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2019-12-08 12:39:50.015741920 +0000 2019-12-08T12:42:49.135 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2019-12-08 12:39:50.015741920 +0000 2019-12-08T12:42:49.135 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2019-12-08T12:42:49.152 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2019-12-08T12:42:49.153 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2019-12-08T12:42:49.153 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes (512 B) copied, 0.00030722 s, 1.7 MB/s 2019-12-08T12:42:49.156 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-08T12:42:49.156 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-08T12:42:49.156 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-08T12:42:49.157 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-08T12:42:49.157 INFO:tasks.ceph:Generating config... 2019-12-08T12:42:49.158 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-12-08T12:42:49.158 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-12-08T12:42:49.158 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-12-08T12:42:49.158 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-12-08T12:42:49.158 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] debug ms = 1 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] debug osd = 25 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] debug bluestore = 20 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] debug bluefs = 20 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2019-12-08T12:42:49.159 INFO:tasks.ceph:[osd] debug rocksdb = 10 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] debug journal = 20 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] bdev async discard = True 2019-12-08T12:42:49.160 INFO:tasks.ceph:[osd] osd fast shutdown = False 2019-12-08T12:42:49.161 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2019-12-08T12:42:49.161 INFO:tasks.ceph:[osd] debug filestore = 20 2019-12-08T12:42:49.161 INFO:tasks.ceph:[osd] bdev enable discard = True 2019-12-08T12:42:49.161 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2019-12-08T12:42:49.161 INFO:tasks.ceph:[mon] debug mon = 20 2019-12-08T12:42:49.162 INFO:tasks.ceph:[mon] debug paxos = 20 2019-12-08T12:42:49.162 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-12-08T12:42:49.162 INFO:tasks.ceph:[mon] debug ms = 1 2019-12-08T12:42:49.162 INFO:tasks.ceph:[client] debug rgw = 20 2019-12-08T12:42:49.162 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-12-08T12:42:49.162 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-12-08T12:42:49.162 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-12-08T12:42:49.162 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-12-08T12:42:49.163 INFO:tasks.ceph:Setting up mon.a... 2019-12-08T12:42:49.163 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-12-08T12:42:49.330 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.keyring 2019-12-08T12:42:49.334 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-12-08T12:42:49.484 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-08T12:42:49.592 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.129'), ('mon.c', '[v2:172.21.15.129:3301,v1:172.21.15.129:6790]'), ('mon.b', '172.21.15.181')] 2019-12-08T12:42:49.592 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.129 --addv c '[v2:172.21.15.129:3301,v1:172.21.15.129:6790]' --add b 172.21.15.181 --print /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:42:49.734 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:42:49.734 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: generated fsid 8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 0 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:fsid 8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:last_changed 2019-12-08T12:42:49.736216+0000 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:created 2019-12-08T12:42:49.736216+0000 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:min_mon_release 0 (unknown) 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2019-12-08T12:42:49.735 INFO:teuthology.orchestra.run.smithi129.stdout:1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2019-12-08T12:42:49.736 INFO:teuthology.orchestra.run.smithi129.stdout:2: [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] mon.c 2019-12-08T12:42:49.736 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-12-08T12:42:49.737 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44... 2019-12-08T12:42:49.738 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-08T12:42:49.741 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-08T12:42:49.959 INFO:tasks.ceph:Creating admin key on mon.a... 2019-12-08T12:42:49.959 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-12-08T12:42:50.121 INFO:tasks.ceph:Copying monmap to all nodes... 2019-12-08T12:42:50.212 DEBUG:teuthology.orchestra.remote:smithi129:/etc/ceph/ceph.keyring is 206B 2019-12-08T12:42:50.371 DEBUG:teuthology.orchestra.remote:smithi129:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-12-08T12:42:50.456 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:42:50.456 INFO:teuthology.orchestra.run.smithi129:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-08T12:42:50.578 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:42:50.644 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:42:50.644 INFO:teuthology.orchestra.run.smithi181:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-08T12:42:50.793 INFO:teuthology.orchestra.run.smithi181:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:42:50.869 INFO:tasks.ceph:Setting up mon nodes... 2019-12-08T12:42:50.869 INFO:tasks.ceph:Setting up mgr nodes... 2019-12-08T12:42:50.870 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-12-08T12:42:51.042 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-12-08T12:42:51.047 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-12-08T12:42:51.249 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-12-08T12:42:51.254 INFO:tasks.ceph:Setting up mds nodes... 2019-12-08T12:42:51.254 INFO:tasks.ceph_client:Setting up client nodes... 2019-12-08T12:42:51.255 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-12-08T12:42:51.407 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-12-08T12:42:51.439 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-12-08T12:42:51.587 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-12-08T12:42:51.619 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-12-08T12:42:51.619 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}} 2019-12-08T12:42:51.619 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-12-08T12:42:51.739 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-08T12:42:51.739 INFO:tasks.ceph:{} 2019-12-08T12:42:51.739 INFO:tasks.ceph:osd.0 2019-12-08T12:42:51.740 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:42:51.740 INFO:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-08T12:42:52.322 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:42:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:42:52.324 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:42:52.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2019-12-08T12:42:52.326 INFO:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-12-08T12:42:52.515 INFO:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-12-08T12:42:52.638 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-12-08T12:42:52.749 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-08T12:42:52.749 INFO:tasks.ceph:{} 2019-12-08T12:42:52.749 INFO:tasks.ceph:osd.1 2019-12-08T12:42:52.749 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:42:52.749 INFO:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-08T12:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:42:53.358 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2019-12-08T12:42:53.358 INFO:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-12-08T12:42:53.493 INFO:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-12-08T12:42:53.599 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-12-08T12:42:53.693 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-08T12:42:53.693 INFO:tasks.ceph:{} 2019-12-08T12:42:53.693 INFO:tasks.ceph:osd.2 2019-12-08T12:42:53.694 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:42:53.694 INFO:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-08T12:42:54.284 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:42:54.284 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:42:54.284 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:42:54.284 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:42:54.285 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:42:54.285 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:42:54.285 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:42:54.285 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:42:54.285 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:42:54.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2019-12-08T12:42:54.286 INFO:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-12-08T12:42:54.461 INFO:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-12-08T12:42:54.543 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-12-08T12:42:54.615 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-08T12:42:54.615 INFO:tasks.ceph:{} 2019-12-08T12:42:54.615 INFO:tasks.ceph:osd.3 2019-12-08T12:42:54.615 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T12:42:54.616 INFO:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-08T12:42:55.142 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:42:55.142 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:42:55.143 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:42:55.143 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:42:55.143 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:42:55.143 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:42:55.143 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:42:55.143 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:42:55.144 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:42:55.145 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2019-12-08T12:42:55.145 INFO:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-12-08T12:42:55.261 INFO:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-12-08T12:42:55.335 INFO:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:42:55.434 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:55.436+0000 7f7102576b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-12-08T12:42:55.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:55.437+0000 7f7102576b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-12-08T12:42:55.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:55.437+0000 7f7102576b40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-08T12:42:55.436 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:55.437+0000 7f7102576b40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-08T12:42:55.436 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:55.437+0000 7f7102576b40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-08T12:42:55.441 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:55.440+0000 7f7102576b40 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2019-12-08T12:42:57.750 INFO:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-12-08T12:42:57.870 INFO:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:42:58.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:58.017+0000 7f4b13dffb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-12-08T12:42:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:58.018+0000 7f4b13dffb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-12-08T12:42:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:58.018+0000 7f4b13dffb40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-08T12:42:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:58.018+0000 7f4b13dffb40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-08T12:42:58.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:58.018+0000 7f4b13dffb40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-08T12:42:58.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:42:58.019+0000 7f4b13dffb40 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2019-12-08T12:43:00.334 INFO:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-12-08T12:43:00.455 INFO:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:00.601 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:00.603+0000 7fba1d80eb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-12-08T12:43:00.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:00.604+0000 7fba1d80eb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-12-08T12:43:00.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:00.604+0000 7fba1d80eb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-08T12:43:00.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:00.604+0000 7fba1d80eb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-08T12:43:00.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:00.604+0000 7fba1d80eb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-08T12:43:00.603 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:00.605+0000 7fba1d80eb40 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2019-12-08T12:43:02.908 INFO:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-12-08T12:43:03.025 INFO:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:03.175+0000 7f71a807bb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-12-08T12:43:03.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:03.176+0000 7f71a807bb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-12-08T12:43:03.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:03.176+0000 7f71a807bb40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-08T12:43:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:03.176+0000 7f71a807bb40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-08T12:43:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:03.176+0000 7f71a807bb40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-08T12:43:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:03.178+0000 7f71a807bb40 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2019-12-08T12:43:05.481 INFO:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-12-08T12:43:05.600 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-12-08T12:43:05.717 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-08T12:43:05.717 INFO:tasks.ceph:{} 2019-12-08T12:43:05.718 INFO:tasks.ceph:osd.4 2019-12-08T12:43:05.718 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:43:05.718 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-08T12:43:06.320 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:43:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:43:06.320 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:43:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:43:06.321 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:43:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:43:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:43:06.322 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:43:06.322 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:43:06.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2019-12-08T12:43:06.324 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-12-08T12:43:06.510 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-12-08T12:43:06.631 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-12-08T12:43:06.745 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-08T12:43:06.746 INFO:tasks.ceph:{} 2019-12-08T12:43:06.746 INFO:tasks.ceph:osd.5 2019-12-08T12:43:06.746 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:43:06.746 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-08T12:43:07.357 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:43:07.358 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:43:07.358 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:43:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:43:07.358 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:43:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:43:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:43:07.359 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:43:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:43:07.360 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2019-12-08T12:43:07.360 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-12-08T12:43:07.522 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-12-08T12:43:07.644 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-12-08T12:43:07.758 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-08T12:43:07.759 INFO:tasks.ceph:{} 2019-12-08T12:43:07.759 INFO:tasks.ceph:osd.6 2019-12-08T12:43:07.759 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:43:07.759 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-08T12:43:08.369 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:43:08.370 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:43:08.370 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:43:08.370 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:43:08.370 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:43:08.370 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:43:08.370 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:43:08.371 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:43:08.371 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:43:08.372 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2019-12-08T12:43:08.372 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-12-08T12:43:08.547 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-12-08T12:43:08.661 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-12-08T12:43:08.758 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-08T12:43:08.758 INFO:tasks.ceph:{} 2019-12-08T12:43:08.758 INFO:tasks.ceph:osd.7 2019-12-08T12:43:08.758 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T12:43:08.759 INFO:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-08T12:43:09.325 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-08T12:43:09.325 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=0, sparse=0 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-08T12:43:09.326 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-08T12:43:09.328 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2019-12-08T12:43:09.328 INFO:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-12-08T12:43:09.475 INFO:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-12-08T12:43:09.583 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:09.738+0000 7f1f703e4b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-12-08T12:43:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:09.742+0000 7f1f703e4b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-12-08T12:43:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:09.742+0000 7f1f703e4b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-08T12:43:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:09.742+0000 7f1f703e4b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-08T12:43:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:09.742+0000 7f1f703e4b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-08T12:43:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:09.750+0000 7f1f703e4b40 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2019-12-08T12:43:12.038 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-12-08T12:43:12.156 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:12.305 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:12.323+0000 7fed41449b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-12-08T12:43:12.306 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:12.324+0000 7fed41449b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-12-08T12:43:12.306 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:12.324+0000 7fed41449b40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-08T12:43:12.306 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:12.324+0000 7fed41449b40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-08T12:43:12.306 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:12.324+0000 7fed41449b40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-08T12:43:12.307 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:12.325+0000 7fed41449b40 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2019-12-08T12:43:14.612 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-12-08T12:43:14.730 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:14.866 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:14.884+0000 7fb25a986b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-12-08T12:43:14.867 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:14.885+0000 7fb25a986b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-12-08T12:43:14.867 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:14.885+0000 7fb25a986b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-08T12:43:14.867 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:14.885+0000 7fb25a986b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-08T12:43:14.868 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:14.885+0000 7fb25a986b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-08T12:43:14.868 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:14.886+0000 7fb25a986b40 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2019-12-08T12:43:17.173 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-12-08T12:43:17.292 INFO:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:17.431 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:17.448+0000 7f72d903bb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-12-08T12:43:17.432 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:17.449+0000 7f72d903bb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-12-08T12:43:17.432 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:17.449+0000 7f72d903bb40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-08T12:43:17.432 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:17.449+0000 7f72d903bb40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-08T12:43:17.433 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:17.449+0000 7f72d903bb40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-08T12:43:17.433 INFO:teuthology.orchestra.run.smithi181.stderr:2019-12-08T12:43:17.450+0000 7f72d903bb40 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2019-12-08T12:43:19.746 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-12-08T12:43:19.862 INFO:tasks.ceph:Reading keys from all nodes... 2019-12-08T12:43:19.863 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:19.954 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.MBxQsLPuvF 2019-12-08T12:43:19.954 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.MBxQsLPuvF 2019-12-08T12:43:20.059 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.MBxQsLPuvF 2019-12-08T12:43:20.229 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.MBxQsLPuvF is 56B 2019-12-08T12:43:20.302 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.MBxQsLPuvF 2019-12-08T12:43:20.377 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:20.449 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.1rvuGtciFy 2019-12-08T12:43:20.449 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.1rvuGtciFy 2019-12-08T12:43:20.563 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.1rvuGtciFy 2019-12-08T12:43:20.760 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.1rvuGtciFy is 56B 2019-12-08T12:43:20.850 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.1rvuGtciFy 2019-12-08T12:43:20.922 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:21.002 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.RVNF3iZ6OR 2019-12-08T12:43:21.002 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.RVNF3iZ6OR 2019-12-08T12:43:21.102 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.RVNF3iZ6OR 2019-12-08T12:43:21.257 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.RVNF3iZ6OR is 56B 2019-12-08T12:43:21.325 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.RVNF3iZ6OR 2019-12-08T12:43:21.382 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:21.509 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.ddzBxQYv1d 2019-12-08T12:43:21.510 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.ddzBxQYv1d 2019-12-08T12:43:21.613 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.ddzBxQYv1d 2019-12-08T12:43:21.799 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.ddzBxQYv1d is 56B 2019-12-08T12:43:21.875 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.ddzBxQYv1d 2019-12-08T12:43:21.936 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:22.058 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.TtqwgrmntX 2019-12-08T12:43:22.058 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.TtqwgrmntX 2019-12-08T12:43:22.152 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.TtqwgrmntX 2019-12-08T12:43:22.351 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.TtqwgrmntX is 56B 2019-12-08T12:43:22.424 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.TtqwgrmntX 2019-12-08T12:43:22.504 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:22.593 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.7vycWDeu77 2019-12-08T12:43:22.593 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.7vycWDeu77 2019-12-08T12:43:22.706 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.7vycWDeu77 2019-12-08T12:43:22.902 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.7vycWDeu77 is 56B 2019-12-08T12:43:22.980 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.7vycWDeu77 2019-12-08T12:43:23.056 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:23.139 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.ozhBlDxEtS 2019-12-08T12:43:23.140 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.ozhBlDxEtS 2019-12-08T12:43:23.246 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.ozhBlDxEtS 2019-12-08T12:43:23.424 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.ozhBlDxEtS is 56B 2019-12-08T12:43:23.489 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.ozhBlDxEtS 2019-12-08T12:43:23.543 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:23.592 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.IqWUNB3sNf 2019-12-08T12:43:23.593 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.IqWUNB3sNf 2019-12-08T12:43:23.662 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.IqWUNB3sNf 2019-12-08T12:43:23.778 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.IqWUNB3sNf is 56B 2019-12-08T12:43:23.834 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.IqWUNB3sNf 2019-12-08T12:43:23.882 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:23.988 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.vs2O8lyDyt 2019-12-08T12:43:23.988 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.vs2O8lyDyt 2019-12-08T12:43:24.070 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.vs2O8lyDyt 2019-12-08T12:43:24.206 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.vs2O8lyDyt is 56B 2019-12-08T12:43:24.264 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.vs2O8lyDyt 2019-12-08T12:43:24.315 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:24.364 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.Gmm8UT34kA 2019-12-08T12:43:24.364 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.Gmm8UT34kA 2019-12-08T12:43:24.429 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.Gmm8UT34kA 2019-12-08T12:43:24.538 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.Gmm8UT34kA is 56B 2019-12-08T12:43:24.588 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.Gmm8UT34kA 2019-12-08T12:43:24.732 DEBUG:teuthology.orchestra.remote:smithi129:/etc/ceph/ceph.client.0.keyring is 59B 2019-12-08T12:43:24.898 DEBUG:teuthology.orchestra.remote:smithi181:/etc/ceph/ceph.client.1.keyring is 59B 2019-12-08T12:43:24.988 INFO:tasks.ceph:Adding keys to all mons... 2019-12-08T12:43:24.989 INFO:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-08T12:43:24.992 INFO:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-08T12:43:25.103 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2019-12-08T12:43:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDL7+xd5n+2AhAANfOeUl+E/VH88JpnCL2Gfg== 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDP7+xdJ0MSGhAAPPy/FUFdzZuPSvbXMcs1xQ== 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDS7+xdZJwZARAAzYE5Roe56cHLYLrGNHwBtA== 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDU7+xd7EcFJBAAjhDfCDzPPtw5YSmcoIm52Q== 2019-12-08T12:43:25.104 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDX7+xdCyCJChAAS0eerSpZp1MWOp4DGNzELg== 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDL7+xdcibhDxAA3v6fvwogWx3adcf+94LZDA== 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDd7+xdxMgOLBAAy11Zy5YeKUJ2fCvwMKz33g== 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDg7+xdMcFUExAALqmKbXTFkpAGf2kaT7rz9w== 2019-12-08T12:43:25.105 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDi7+xd/6HONBAA3Q3V8CkUrFRNGSYJ71qECg== 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDl7+xdtqbfGhAA3I0Fi0hSE2vBfK317UElGA== 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDL7+xdkHBuGBAAO0HrAWGBDsPkODg6QqoXMA== 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2019-12-08T12:43:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDL7+xdxoD3IxAAY6i+u9q3Z3ssvVEv4dXglA== 2019-12-08T12:43:25.108 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.y] 2019-12-08T12:43:25.108 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDL7+xd5n+2AhAANfOeUl+E/VH88JpnCL2Gfg== 2019-12-08T12:43:25.108 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2019-12-08T12:43:25.108 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDP7+xdJ0MSGhAAPPy/FUFdzZuPSvbXMcs1xQ== 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDS7+xdZJwZARAAzYE5Roe56cHLYLrGNHwBtA== 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDU7+xd7EcFJBAAjhDfCDzPPtw5YSmcoIm52Q== 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDX7+xdCyCJChAAS0eerSpZp1MWOp4DGNzELg== 2019-12-08T12:43:25.109 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2019-12-08T12:43:25.110 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDL7+xdcibhDxAA3v6fvwogWx3adcf+94LZDA== 2019-12-08T12:43:25.110 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2019-12-08T12:43:25.110 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDd7+xdxMgOLBAAy11Zy5YeKUJ2fCvwMKz33g== 2019-12-08T12:43:25.110 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2019-12-08T12:43:25.110 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDg7+xdMcFUExAALqmKbXTFkpAGf2kaT7rz9w== 2019-12-08T12:43:25.110 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.6] 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDi7+xd/6HONBAA3Q3V8CkUrFRNGSYJ71qECg== 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.7] 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDl7+xdtqbfGhAA3I0Fi0hSE2vBfK317UElGA== 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDL7+xdkHBuGBAAO0HrAWGBDsPkODg6QqoXMA== 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2019-12-08T12:43:25.111 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDL7+xdxoD3IxAAY6i+u9q3Z3ssvVEv4dXglA== 2019-12-08T12:43:25.112 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-08T12:43:25.115 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-08T12:43:25.284 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.287 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.450 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.453 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.626 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.630 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.791 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.794 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:25.945 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-08T12:43:25.948 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-08T12:43:26.084 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.088 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.241 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.245 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.388 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.392 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.547 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.551 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-08T12:43:26.715 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-08T12:43:26.720 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-08T12:43:26.886 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-08T12:43:26.890 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-08T12:43:27.063 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-12-08T12:43:27.063 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-12-08T12:43:27.181 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-08T12:43:27.345 INFO:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-12-08T12:43:27.464 INFO:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-12-08T12:43:27.580 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-08T12:43:27.738 INFO:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-12-08T12:43:27.842 INFO:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-12-08T12:43:27.961 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-08T12:43:28.149 INFO:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-12-08T12:43:28.261 INFO:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:28.268 INFO:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-08T12:43:28.353 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-12-08T12:43:28.353 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-08T12:43:28.353 INFO:tasks.ceph.mon.a:Restarting daemon 2019-12-08T12:43:28.354 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-12-08T12:43:28.357 INFO:tasks.ceph.mon.a:Started 2019-12-08T12:43:28.357 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-08T12:43:28.357 INFO:tasks.ceph.mon.c:Restarting daemon 2019-12-08T12:43:28.358 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-12-08T12:43:28.361 INFO:tasks.ceph.mon.c:Started 2019-12-08T12:43:28.361 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-08T12:43:28.361 INFO:tasks.ceph.mon.b:Restarting daemon 2019-12-08T12:43:28.361 INFO:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-12-08T12:43:28.365 INFO:tasks.ceph.mon.b:Started 2019-12-08T12:43:28.365 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-12-08T12:43:28.365 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-12-08T12:43:28.365 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-12-08T12:43:28.368 INFO:tasks.ceph.mgr.y:Started 2019-12-08T12:43:28.368 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-12-08T12:43:28.369 INFO:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-12-08T12:43:28.371 INFO:tasks.ceph.mgr.x:Started 2019-12-08T12:43:28.371 INFO:tasks.ceph:Setting crush tunables to default 2019-12-08T12:43:28.372 INFO:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd crush tunables default 2019-12-08T12:43:32.413 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:00:03.845 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:43:32.449 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:00:03.880 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:43:32.633 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:00:04.073 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:43:44.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:43:44.518+0000 7f8db759e700 0 --2- 172.21.15.129:0/1675154373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d98009310 0x7f8d9800b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-08T12:43:45.585 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2019-12-08T12:43:45.598 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-12-08T12:43:45.598 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:45.646 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.xKERH5lg6i 2019-12-08T12:43:45.646 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.xKERH5lg6i 2019-12-08T12:43:45.712 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.xKERH5lg6i 2019-12-08T12:43:45.811 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.xKERH5lg6i is 37B 2019-12-08T12:43:45.857 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.xKERH5lg6i 2019-12-08T12:43:45.900 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:45.994 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.KP4S8D2BKf 2019-12-08T12:43:45.994 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.KP4S8D2BKf 2019-12-08T12:43:46.062 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.KP4S8D2BKf 2019-12-08T12:43:46.163 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.KP4S8D2BKf is 37B 2019-12-08T12:43:46.227 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.KP4S8D2BKf 2019-12-08T12:43:46.271 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:46.360 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.CjhNQPAuqt 2019-12-08T12:43:46.360 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.CjhNQPAuqt 2019-12-08T12:43:46.417 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.CjhNQPAuqt 2019-12-08T12:43:46.517 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.CjhNQPAuqt is 37B 2019-12-08T12:43:46.564 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.CjhNQPAuqt 2019-12-08T12:43:46.606 INFO:teuthology.orchestra.run.smithi129:> mktemp 2019-12-08T12:43:46.699 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.vtWM6CVUdm 2019-12-08T12:43:46.699 INFO:teuthology.orchestra.run.smithi129:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.vtWM6CVUdm 2019-12-08T12:43:46.754 INFO:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.vtWM6CVUdm 2019-12-08T12:43:46.851 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.vtWM6CVUdm is 37B 2019-12-08T12:43:46.898 INFO:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.vtWM6CVUdm 2019-12-08T12:43:46.938 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:47.028 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.XTyojBG7RO 2019-12-08T12:43:47.029 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.XTyojBG7RO 2019-12-08T12:43:47.129 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.XTyojBG7RO 2019-12-08T12:43:47.278 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.XTyojBG7RO is 37B 2019-12-08T12:43:47.336 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.XTyojBG7RO 2019-12-08T12:43:47.388 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:47.513 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.Flk2vSRmJF 2019-12-08T12:43:47.514 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.Flk2vSRmJF 2019-12-08T12:43:47.590 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.Flk2vSRmJF 2019-12-08T12:43:47.698 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.Flk2vSRmJF is 37B 2019-12-08T12:43:47.749 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.Flk2vSRmJF 2019-12-08T12:43:47.792 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:47.896 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.yHHkrdVAga 2019-12-08T12:43:47.897 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.yHHkrdVAga 2019-12-08T12:43:47.974 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.yHHkrdVAga 2019-12-08T12:43:48.093 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.yHHkrdVAga is 37B 2019-12-08T12:43:48.147 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.yHHkrdVAga 2019-12-08T12:43:48.198 INFO:teuthology.orchestra.run.smithi181:> mktemp 2019-12-08T12:43:48.245 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.DZYF2DJHM1 2019-12-08T12:43:48.245 INFO:teuthology.orchestra.run.smithi181:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.DZYF2DJHM1 2019-12-08T12:43:48.306 INFO:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.DZYF2DJHM1 2019-12-08T12:43:48.424 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.DZYF2DJHM1 is 37B 2019-12-08T12:43:48.477 INFO:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.DZYF2DJHM1 2019-12-08T12:43:48.520 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 8f208783-5139-4d48-8ba7-aa9049936271 0 2019-12-08T12:43:49.738 INFO:teuthology.orchestra.run.smithi181.stdout:0 2019-12-08T12:43:49.753 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new cc8d989c-3ca6-4568-8bf3-c4550e51e8cf 1 2019-12-08T12:43:51.223 INFO:teuthology.orchestra.run.smithi181.stdout:1 2019-12-08T12:43:51.246 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 708c0e93-0abe-42db-a152-1c035b1275d3 2 2019-12-08T12:43:52.328 INFO:teuthology.orchestra.run.smithi181.stdout:2 2019-12-08T12:43:52.350 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new fdc80253-76ed-4781-b47b-c152c1226e9b 3 2019-12-08T12:43:53.484 INFO:teuthology.orchestra.run.smithi181.stdout:3 2019-12-08T12:43:53.507 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1 4 2019-12-08T12:43:54.666 INFO:teuthology.orchestra.run.smithi181.stdout:4 2019-12-08T12:43:54.693 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new c9455e5c-2b38-4d12-9940-3d3e62337493 5 2019-12-08T12:43:55.771 INFO:teuthology.orchestra.run.smithi181.stdout:5 2019-12-08T12:43:55.793 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e 6 2019-12-08T12:43:56.997 INFO:teuthology.orchestra.run.smithi181.stdout:6 2019-12-08T12:43:57.023 INFO:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new fea090fa-cc6a-460a-8b78-ff8111552350 7 2019-12-08T12:43:58.047 INFO:teuthology.orchestra.run.smithi181.stdout:7 2019-12-08T12:43:58.070 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.071 INFO:tasks.ceph.osd.0:Restarting daemon 2019-12-08T12:43:58.071 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-12-08T12:43:58.074 INFO:tasks.ceph.osd.0:Started 2019-12-08T12:43:58.074 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.074 INFO:tasks.ceph.osd.1:Restarting daemon 2019-12-08T12:43:58.075 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-12-08T12:43:58.077 INFO:tasks.ceph.osd.1:Started 2019-12-08T12:43:58.078 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.078 INFO:tasks.ceph.osd.2:Restarting daemon 2019-12-08T12:43:58.078 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-12-08T12:43:58.081 INFO:tasks.ceph.osd.2:Started 2019-12-08T12:43:58.081 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.081 INFO:tasks.ceph.osd.3:Restarting daemon 2019-12-08T12:43:58.081 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-12-08T12:43:58.084 INFO:tasks.ceph.osd.3:Started 2019-12-08T12:43:58.084 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.085 INFO:tasks.ceph.osd.4:Restarting daemon 2019-12-08T12:43:58.085 INFO:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-12-08T12:43:58.088 INFO:tasks.ceph.osd.4:Started 2019-12-08T12:43:58.088 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.089 INFO:tasks.ceph.osd.5:Restarting daemon 2019-12-08T12:43:58.089 INFO:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-12-08T12:43:58.092 INFO:tasks.ceph.osd.5:Started 2019-12-08T12:43:58.092 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.092 INFO:tasks.ceph.osd.6:Restarting daemon 2019-12-08T12:43:58.093 INFO:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-12-08T12:43:58.096 INFO:tasks.ceph.osd.6:Started 2019-12-08T12:43:58.096 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-08T12:43:58.096 INFO:tasks.ceph.osd.7:Restarting daemon 2019-12-08T12:43:58.097 INFO:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-12-08T12:43:58.101 INFO:tasks.ceph.osd.7:Started 2019-12-08T12:43:58.101 INFO:tasks.ceph:Waiting for OSDs to come up 2019-12-08T12:43:58.909 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-08T12:43:59.951 INFO:teuthology.misc.health.smithi129.stdout: 2019-12-08T12:43:59.951 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":10,"fsid":"8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44","created":"2019-12-08T12:43:42.199729+0000","modified":"2019-12-08T12:43:57.914586+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"8f208783-5139-4d48-8ba7-aa9049936271","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cc8d989c-3ca6-4568-8bf3-c4550e51e8cf","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"708c0e93-0abe-42db-a152-1c035b1275d3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fdc80253-76ed-4781-b47b-c152c1226e9b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c9455e5c-2b38-4d12-9940-3d3e62337493","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fea090fa-cc6a-460a-8b78-ff8111552350","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-08T12:43:59.965 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-08T12:44:01.883 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:03.647 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.888 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:03.651 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.910 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:03.672 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.910 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:03.713 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.930 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:03.693 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.940 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:03.703 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.944 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:03.707 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.966 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:03.763 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.967 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:03.729 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.971 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:03.773 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.985 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:03.785 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:01.986 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:03.749 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.023 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:03.820 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.066 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:03.866 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.200 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:03.992 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.257 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:04.049 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T12:44:02.560+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.563 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T12:44:02.569+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.568 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T12:44:02.576+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.581 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T12:44:02.559+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.595 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T12:44:02.601+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.607 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T12:44:02.628+0000 114ae700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6822/0: (98) Address already in use 2019-12-08T12:44:02.621 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T12:44:02.606+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.629 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T12:44:02.620+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.751 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:04.514 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.761 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:04.525 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.783 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:04.546 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.821 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:04.584 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.842 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:04.639 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.860 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:04.663 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:02.869 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T12:44:02.850+0000 9b13800 -1 Falling back to public interface 2019-12-08T12:44:02.907 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:04.707 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.081 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:04.873 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.122 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:04.885 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.134 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:04.897 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.138 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:04.902 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.150 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:04.913 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.176 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:04.939 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.188 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:04.951 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.224 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:04.980 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.228 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:04.992 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.239 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:05.036 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.251 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:05.049 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.279 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:05.077 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.288 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:05.090 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.289 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:05.089 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.300 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:05.102 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.463 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:05.255 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:03.474 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:05.267 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:05.906 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:07.669 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:05.912 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:07.675 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.178 INFO:tasks.ceph.osd.7.smithi181.stderr:==00:00:00:07.941 10375== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.215 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:07.978 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.221 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:07.984 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.227 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:07.990 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.233 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:07.996 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.238 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:08.035 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.248 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:08.046 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.279 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:08.042 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.288 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:08.051 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.397 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:08.189 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.403 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:08.195 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.488 INFO:tasks.ceph.osd.4.smithi181.stderr:==00:00:00:08.251 10373== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.500 INFO:tasks.ceph.osd.6.smithi181.stderr:==00:00:00:08.263 10374== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.512 INFO:tasks.ceph.osd.1.smithi129.stderr:==00:00:00:08.309 10197== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.549 INFO:tasks.ceph.osd.5.smithi181.stderr:==00:00:00:08.312 10376== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.645 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:08.445 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.650 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:08.451 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.666 INFO:tasks.ceph.osd.3.smithi129.stderr:==00:00:00:08.459 10198== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.748 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:08.550 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.757 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:08.559 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:06.770 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-08T12:44:06.915 INFO:tasks.ceph.osd.0.smithi129.stderr:==00:00:00:08.715 10196== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:07.019 INFO:tasks.ceph.osd.2.smithi129.stderr:==00:00:00:08.821 10195== Warning: unimplemented fcntl command: 1038 2019-12-08T12:44:07.519 INFO:teuthology.misc.health.smithi129.stdout: 2019-12-08T12:44:07.520 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":10,"fsid":"8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44","created":"2019-12-08T12:43:42.199729+0000","modified":"2019-12-08T12:43:57.914586+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"8f208783-5139-4d48-8ba7-aa9049936271","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cc8d989c-3ca6-4568-8bf3-c4550e51e8cf","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"708c0e93-0abe-42db-a152-1c035b1275d3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fdc80253-76ed-4781-b47b-c152c1226e9b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c9455e5c-2b38-4d12-9940-3d3e62337493","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fea090fa-cc6a-460a-8b78-ff8111552350","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-08T12:44:07.534 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-08T12:44:07.653 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T12:44:07.674+0000 9b13800 -1 osd.7 0 log_to_monitors {default=true} 2019-12-08T12:44:07.910 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T12:44:07.931+0000 9b13800 -1 osd.6 0 log_to_monitors {default=true} 2019-12-08T12:44:07.912 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T12:44:07.931+0000 9b13800 -1 osd.4 0 log_to_monitors {default=true} 2019-12-08T12:44:07.926 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T12:44:07.947+0000 9b13800 -1 osd.5 0 log_to_monitors {default=true} 2019-12-08T12:44:07.959 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T12:44:07.959+0000 9b13800 -1 osd.1 0 log_to_monitors {default=true} 2019-12-08T12:44:08.122 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T12:44:08.123+0000 9b13800 -1 osd.3 0 log_to_monitors {default=true} 2019-12-08T12:44:08.331 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T12:44:08.331+0000 9b13800 -1 osd.0 0 log_to_monitors {default=true} 2019-12-08T12:44:08.460 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T12:44:08.461+0000 9b13800 -1 osd.2 0 log_to_monitors {default=true} 2019-12-08T12:44:10.017 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T12:44:10.016+0000 2476e700 -1 osd.3 0 waiting for initial osdmap 2019-12-08T12:44:10.082 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T12:44:10.097+0000 2476e700 -1 osd.6 0 waiting for initial osdmap 2019-12-08T12:44:10.107 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T12:44:10.106+0000 2476e700 -1 osd.0 0 waiting for initial osdmap 2019-12-08T12:44:10.119 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T12:44:10.119+0000 2476e700 -1 osd.1 0 waiting for initial osdmap 2019-12-08T12:44:10.229 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T12:44:10.244+0000 27772700 -1 osd.4 0 waiting for initial osdmap 2019-12-08T12:44:10.239 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T12:44:10.257+0000 27772700 -1 osd.7 0 waiting for initial osdmap 2019-12-08T12:44:10.240 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T12:44:10.257+0000 27772700 -1 osd.5 0 waiting for initial osdmap 2019-12-08T12:44:11.337 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T12:44:11.332+0000 2476e700 -1 osd.2 0 waiting for initial osdmap 2019-12-08T12:44:14.340 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-08T12:44:15.006 INFO:teuthology.misc.health.smithi129.stdout: 2019-12-08T12:44:15.007 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":16,"fsid":"8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44","created":"2019-12-08T12:43:42.199729+0000","modified":"2019-12-08T12:44:14.089234+0000","last_up_change":"2019-12-08T12:44:12.010706+0000","last_in_change":"2019-12-08T12:44:12.010706+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"8f208783-5139-4d48-8ba7-aa9049936271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":10196},{"type":"v1","addr":"172.21.15.129:6811","nonce":10196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":10196},{"type":"v1","addr":"172.21.15.129:6815","nonce":10196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":10196},{"type":"v1","addr":"172.21.15.129:6823","nonce":10196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":10196},{"type":"v1","addr":"172.21.15.129:6818","nonce":10196}]},"public_addr":"172.21.15.129:6811/10196","cluster_addr":"172.21.15.129:6815/10196","heartbeat_back_addr":"172.21.15.129:6823/10196","heartbeat_front_addr":"172.21.15.129:6818/10196","state":["exists","up"]},{"osd":1,"uuid":"cc8d989c-3ca6-4568-8bf3-c4550e51e8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":10197},{"type":"v1","addr":"172.21.15.129:6813","nonce":10197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6817","nonce":10197},{"type":"v1","addr":"172.21.15.129:6819","nonce":10197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":10197},{"type":"v1","addr":"172.21.15.129:6825","nonce":10197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":10197},{"type":"v1","addr":"172.21.15.129:6822","nonce":10197}]},"public_addr":"172.21.15.129:6813/10197","cluster_addr":"172.21.15.129:6819/10197","heartbeat_back_addr":"172.21.15.129:6825/10197","heartbeat_front_addr":"172.21.15.129:6822/10197","state":["exists","up"]},{"osd":2,"uuid":"708c0e93-0abe-42db-a152-1c035b1275d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":10195},{"type":"v1","addr":"172.21.15.129:6803","nonce":10195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":10195},{"type":"v1","addr":"172.21.15.129:6805","nonce":10195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":10195},{"type":"v1","addr":"172.21.15.129:6809","nonce":10195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":10195},{"type":"v1","addr":"172.21.15.129:6807","nonce":10195}]},"public_addr":"172.21.15.129:6803/10195","cluster_addr":"172.21.15.129:6805/10195","heartbeat_back_addr":"172.21.15.129:6809/10195","heartbeat_front_addr":"172.21.15.129:6807/10195","state":["exists","up"]},{"osd":3,"uuid":"fdc80253-76ed-4781-b47b-c152c1226e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":10198},{"type":"v1","addr":"172.21.15.129:6827","nonce":10198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":10198},{"type":"v1","addr":"172.21.15.129:6829","nonce":10198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":10198},{"type":"v1","addr":"172.21.15.129:6833","nonce":10198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":10198},{"type":"v1","addr":"172.21.15.129:6831","nonce":10198}]},"public_addr":"172.21.15.129:6827/10198","cluster_addr":"172.21.15.129:6829/10198","heartbeat_back_addr":"172.21.15.129:6833/10198","heartbeat_front_addr":"172.21.15.129:6831/10198","state":["exists","up"]},{"osd":4,"uuid":"0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":10373},{"type":"v1","addr":"172.21.15.181:6801","nonce":10373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":10373},{"type":"v1","addr":"172.21.15.181:6805","nonce":10373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":10373},{"type":"v1","addr":"172.21.15.181:6820","nonce":10373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":10373},{"type":"v1","addr":"172.21.15.181:6812","nonce":10373}]},"public_addr":"172.21.15.181:6801/10373","cluster_addr":"172.21.15.181:6805/10373","heartbeat_back_addr":"172.21.15.181:6820/10373","heartbeat_front_addr":"172.21.15.181:6812/10373","state":["exists","up"]},{"osd":5,"uuid":"c9455e5c-2b38-4d12-9940-3d3e62337493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":10376},{"type":"v1","addr":"172.21.15.181:6803","nonce":10376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":10376},{"type":"v1","addr":"172.21.15.181:6808","nonce":10376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":10376},{"type":"v1","addr":"172.21.15.181:6818","nonce":10376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":10376},{"type":"v1","addr":"172.21.15.181:6813","nonce":10376}]},"public_addr":"172.21.15.181:6803/10376","cluster_addr":"172.21.15.181:6808/10376","heartbeat_back_addr":"172.21.15.181:6818/10376","heartbeat_front_addr":"172.21.15.181:6813/10376","state":["exists","up"]},{"osd":6,"uuid":"2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":10374},{"type":"v1","addr":"172.21.15.181:6825","nonce":10374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":10374},{"type":"v1","addr":"172.21.15.181:6827","nonce":10374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":10374},{"type":"v1","addr":"172.21.15.181:6831","nonce":10374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":10374},{"type":"v1","addr":"172.21.15.181:6829","nonce":10374}]},"public_addr":"172.21.15.181:6825/10374","cluster_addr":"172.21.15.181:6827/10374","heartbeat_back_addr":"172.21.15.181:6831/10374","heartbeat_front_addr":"172.21.15.181:6829/10374","state":["exists","up"]},{"osd":7,"uuid":"fea090fa-cc6a-460a-8b78-ff8111552350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":10375},{"type":"v1","addr":"172.21.15.181:6809","nonce":10375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":10375},{"type":"v1","addr":"172.21.15.181:6817","nonce":10375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":10375},{"type":"v1","addr":"172.21.15.181:6824","nonce":10375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":10375},{"type":"v1","addr":"172.21.15.181:6821","nonce":10375}]},"public_addr":"172.21.15.181:6809/10375","cluster_addr":"172.21.15.181:6817/10375","heartbeat_back_addr":"172.21.15.181:6824/10375","heartbeat_front_addr":"172.21.15.181:6821/10375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.409855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.091424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.567144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.187006+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.097427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.114663+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.989105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.848599+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-08T12:44:15.019 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-12-08T12:44:15.019 INFO:tasks.ceph:Creating RBD pool 2019-12-08T12:44:15.020 INFO:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd pool create rbd 8 2019-12-08T12:44:16.978 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'rbd' created 2019-12-08T12:44:16.994 INFO:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-12-08T12:44:18.008 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rbd' on pool 'rbd' 2019-12-08T12:44:18.022 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-12-08T12:44:18.023 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-08T12:44:18.023 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-12-08T12:44:18.743 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:18.743 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":18,"fsid":"8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44","created":"2019-12-08T12:43:42.199729+0000","modified":"2019-12-08T12:44:17.850402+0000","last_up_change":"2019-12-08T12:44:12.010706+0000","last_in_change":"2019-12-08T12:44:12.010706+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-08T12:44:15.859276+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8f208783-5139-4d48-8ba7-aa9049936271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":10196},{"type":"v1","addr":"172.21.15.129:6811","nonce":10196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":10196},{"type":"v1","addr":"172.21.15.129:6815","nonce":10196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":10196},{"type":"v1","addr":"172.21.15.129:6823","nonce":10196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":10196},{"type":"v1","addr":"172.21.15.129:6818","nonce":10196}]},"public_addr":"172.21.15.129:6811/10196","cluster_addr":"172.21.15.129:6815/10196","heartbeat_back_addr":"172.21.15.129:6823/10196","heartbeat_front_addr":"172.21.15.129:6818/10196","state":["exists","up"]},{"osd":1,"uuid":"cc8d989c-3ca6-4568-8bf3-c4550e51e8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":10197},{"type":"v1","addr":"172.21.15.129:6813","nonce":10197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6817","nonce":10197},{"type":"v1","addr":"172.21.15.129:6819","nonce":10197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":10197},{"type":"v1","addr":"172.21.15.129:6825","nonce":10197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":10197},{"type":"v1","addr":"172.21.15.129:6822","nonce":10197}]},"public_addr":"172.21.15.129:6813/10197","cluster_addr":"172.21.15.129:6819/10197","heartbeat_back_addr":"172.21.15.129:6825/10197","heartbeat_front_addr":"172.21.15.129:6822/10197","state":["exists","up"]},{"osd":2,"uuid":"708c0e93-0abe-42db-a152-1c035b1275d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":10195},{"type":"v1","addr":"172.21.15.129:6803","nonce":10195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":10195},{"type":"v1","addr":"172.21.15.129:6805","nonce":10195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":10195},{"type":"v1","addr":"172.21.15.129:6809","nonce":10195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":10195},{"type":"v1","addr":"172.21.15.129:6807","nonce":10195}]},"public_addr":"172.21.15.129:6803/10195","cluster_addr":"172.21.15.129:6805/10195","heartbeat_back_addr":"172.21.15.129:6809/10195","heartbeat_front_addr":"172.21.15.129:6807/10195","state":["exists","up"]},{"osd":3,"uuid":"fdc80253-76ed-4781-b47b-c152c1226e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":10198},{"type":"v1","addr":"172.21.15.129:6827","nonce":10198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":10198},{"type":"v1","addr":"172.21.15.129:6829","nonce":10198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":10198},{"type":"v1","addr":"172.21.15.129:6833","nonce":10198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":10198},{"type":"v1","addr":"172.21.15.129:6831","nonce":10198}]},"public_addr":"172.21.15.129:6827/10198","cluster_addr":"172.21.15.129:6829/10198","heartbeat_back_addr":"172.21.15.129:6833/10198","heartbeat_front_addr":"172.21.15.129:6831/10198","state":["exists","up"]},{"osd":4,"uuid":"0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":10373},{"type":"v1","addr":"172.21.15.181:6801","nonce":10373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":10373},{"type":"v1","addr":"172.21.15.181:6805","nonce":10373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":10373},{"type":"v1","addr":"172.21.15.181:6820","nonce":10373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":10373},{"type":"v1","addr":"172.21.15.181:6812","nonce":10373}]},"public_addr":"172.21.15.181:6801/10373","cluster_addr":"172.21.15.181:6805/10373","heartbeat_back_addr":"172.21.15.181:6820/10373","heartbeat_front_addr":"172.21.15.181:6812/10373","state":["exists","up"]},{"osd":5,"uuid":"c9455e5c-2b38-4d12-9940-3d3e62337493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":10376},{"type":"v1","addr":"172.21.15.181:6803","nonce":10376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":10376},{"type":"v1","addr":"172.21.15.181:6808","nonce":10376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":10376},{"type":"v1","addr":"172.21.15.181:6818","nonce":10376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":10376},{"type":"v1","addr":"172.21.15.181:6813","nonce":10376}]},"public_addr":"172.21.15.181:6803/10376","cluster_addr":"172.21.15.181:6808/10376","heartbeat_back_addr":"172.21.15.181:6818/10376","heartbeat_front_addr":"172.21.15.181:6813/10376","state":["exists","up"]},{"osd":6,"uuid":"2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":10374},{"type":"v1","addr":"172.21.15.181:6825","nonce":10374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":10374},{"type":"v1","addr":"172.21.15.181:6827","nonce":10374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":10374},{"type":"v1","addr":"172.21.15.181:6831","nonce":10374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":10374},{"type":"v1","addr":"172.21.15.181:6829","nonce":10374}]},"public_addr":"172.21.15.181:6825/10374","cluster_addr":"172.21.15.181:6827/10374","heartbeat_back_addr":"172.21.15.181:6831/10374","heartbeat_front_addr":"172.21.15.181:6829/10374","state":["exists","up"]},{"osd":7,"uuid":"fea090fa-cc6a-460a-8b78-ff8111552350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":10375},{"type":"v1","addr":"172.21.15.181:6809","nonce":10375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":10375},{"type":"v1","addr":"172.21.15.181:6817","nonce":10375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":10375},{"type":"v1","addr":"172.21.15.181:6824","nonce":10375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":10375},{"type":"v1","addr":"172.21.15.181:6821","nonce":10375}]},"public_addr":"172.21.15.181:6809/10375","cluster_addr":"172.21.15.181:6817/10375","heartbeat_back_addr":"172.21.15.181:6824/10375","heartbeat_front_addr":"172.21.15.181:6821/10375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.409855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.091424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.567144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.187006+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.097427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.114663+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.989105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.848599+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-08T12:44:18.755 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-12-08T12:44:15.859276+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'18', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-12-08T12:44:18.756 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-12-08T12:44:19.451 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 8 2019-12-08T12:44:19.464 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-12-08T12:44:19.464 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-12-08T12:44:19.464 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-12-08T12:44:20.375 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:20.393 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":9246},{"type":"v1","addr":"172.21.15.129:6801","nonce":9246}]},"active_addr":"172.21.15.129:6801/9246","active_change":"2019-12-08T12:43:46.186513+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-12-08T12:44:20.400 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-12-08T12:44:20.400 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-12-08T12:44:20.400 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-08T12:44:21.250 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:21.250 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44","created":"2019-12-08T12:43:42.199729+0000","modified":"2019-12-08T12:44:20.189949+0000","last_up_change":"2019-12-08T12:44:12.010706+0000","last_in_change":"2019-12-08T12:44:12.010706+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-08T12:44:15.859276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8f208783-5139-4d48-8ba7-aa9049936271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":10196},{"type":"v1","addr":"172.21.15.129:6811","nonce":10196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":10196},{"type":"v1","addr":"172.21.15.129:6815","nonce":10196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":10196},{"type":"v1","addr":"172.21.15.129:6823","nonce":10196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":10196},{"type":"v1","addr":"172.21.15.129:6818","nonce":10196}]},"public_addr":"172.21.15.129:6811/10196","cluster_addr":"172.21.15.129:6815/10196","heartbeat_back_addr":"172.21.15.129:6823/10196","heartbeat_front_addr":"172.21.15.129:6818/10196","state":["exists","up"]},{"osd":1,"uuid":"cc8d989c-3ca6-4568-8bf3-c4550e51e8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":10197},{"type":"v1","addr":"172.21.15.129:6813","nonce":10197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6817","nonce":10197},{"type":"v1","addr":"172.21.15.129:6819","nonce":10197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":10197},{"type":"v1","addr":"172.21.15.129:6825","nonce":10197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":10197},{"type":"v1","addr":"172.21.15.129:6822","nonce":10197}]},"public_addr":"172.21.15.129:6813/10197","cluster_addr":"172.21.15.129:6819/10197","heartbeat_back_addr":"172.21.15.129:6825/10197","heartbeat_front_addr":"172.21.15.129:6822/10197","state":["exists","up"]},{"osd":2,"uuid":"708c0e93-0abe-42db-a152-1c035b1275d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":10195},{"type":"v1","addr":"172.21.15.129:6803","nonce":10195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":10195},{"type":"v1","addr":"172.21.15.129:6805","nonce":10195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":10195},{"type":"v1","addr":"172.21.15.129:6809","nonce":10195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":10195},{"type":"v1","addr":"172.21.15.129:6807","nonce":10195}]},"public_addr":"172.21.15.129:6803/10195","cluster_addr":"172.21.15.129:6805/10195","heartbeat_back_addr":"172.21.15.129:6809/10195","heartbeat_front_addr":"172.21.15.129:6807/10195","state":["exists","up"]},{"osd":3,"uuid":"fdc80253-76ed-4781-b47b-c152c1226e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":10198},{"type":"v1","addr":"172.21.15.129:6827","nonce":10198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":10198},{"type":"v1","addr":"172.21.15.129:6829","nonce":10198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":10198},{"type":"v1","addr":"172.21.15.129:6833","nonce":10198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":10198},{"type":"v1","addr":"172.21.15.129:6831","nonce":10198}]},"public_addr":"172.21.15.129:6827/10198","cluster_addr":"172.21.15.129:6829/10198","heartbeat_back_addr":"172.21.15.129:6833/10198","heartbeat_front_addr":"172.21.15.129:6831/10198","state":["exists","up"]},{"osd":4,"uuid":"0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":10373},{"type":"v1","addr":"172.21.15.181:6801","nonce":10373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":10373},{"type":"v1","addr":"172.21.15.181:6805","nonce":10373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":10373},{"type":"v1","addr":"172.21.15.181:6820","nonce":10373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":10373},{"type":"v1","addr":"172.21.15.181:6812","nonce":10373}]},"public_addr":"172.21.15.181:6801/10373","cluster_addr":"172.21.15.181:6805/10373","heartbeat_back_addr":"172.21.15.181:6820/10373","heartbeat_front_addr":"172.21.15.181:6812/10373","state":["exists","up"]},{"osd":5,"uuid":"c9455e5c-2b38-4d12-9940-3d3e62337493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":10376},{"type":"v1","addr":"172.21.15.181:6803","nonce":10376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":10376},{"type":"v1","addr":"172.21.15.181:6808","nonce":10376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":10376},{"type":"v1","addr":"172.21.15.181:6818","nonce":10376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":10376},{"type":"v1","addr":"172.21.15.181:6813","nonce":10376}]},"public_addr":"172.21.15.181:6803/10376","cluster_addr":"172.21.15.181:6808/10376","heartbeat_back_addr":"172.21.15.181:6818/10376","heartbeat_front_addr":"172.21.15.181:6813/10376","state":["exists","up"]},{"osd":6,"uuid":"2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":10374},{"type":"v1","addr":"172.21.15.181:6825","nonce":10374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":10374},{"type":"v1","addr":"172.21.15.181:6827","nonce":10374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":10374},{"type":"v1","addr":"172.21.15.181:6831","nonce":10374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":10374},{"type":"v1","addr":"172.21.15.181:6829","nonce":10374}]},"public_addr":"172.21.15.181:6825/10374","cluster_addr":"172.21.15.181:6827/10374","heartbeat_back_addr":"172.21.15.181:6831/10374","heartbeat_front_addr":"172.21.15.181:6829/10374","state":["exists","up"]},{"osd":7,"uuid":"fea090fa-cc6a-460a-8b78-ff8111552350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":10375},{"type":"v1","addr":"172.21.15.181:6809","nonce":10375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":10375},{"type":"v1","addr":"172.21.15.181:6817","nonce":10375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":10375},{"type":"v1","addr":"172.21.15.181:6824","nonce":10375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":10375},{"type":"v1","addr":"172.21.15.181:6821","nonce":10375}]},"public_addr":"172.21.15.181:6809/10375","cluster_addr":"172.21.15.181:6817/10375","heartbeat_back_addr":"172.21.15.181:6824/10375","heartbeat_front_addr":"172.21.15.181:6821/10375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.409855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.091424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.567144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.187006+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.097427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.114663+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.989105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.848599+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-08T12:44:21.276 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-12-08T12:44:21.276 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-08T12:44:22.163 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:22.163 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"8c7dd0ae-e1c2-4577-ac68-cf7b4f8fff44","created":"2019-12-08T12:43:42.199729+0000","modified":"2019-12-08T12:44:20.189949+0000","last_up_change":"2019-12-08T12:44:12.010706+0000","last_in_change":"2019-12-08T12:44:12.010706+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-08T12:44:15.859276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8f208783-5139-4d48-8ba7-aa9049936271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":10196},{"type":"v1","addr":"172.21.15.129:6811","nonce":10196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":10196},{"type":"v1","addr":"172.21.15.129:6815","nonce":10196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":10196},{"type":"v1","addr":"172.21.15.129:6823","nonce":10196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":10196},{"type":"v1","addr":"172.21.15.129:6818","nonce":10196}]},"public_addr":"172.21.15.129:6811/10196","cluster_addr":"172.21.15.129:6815/10196","heartbeat_back_addr":"172.21.15.129:6823/10196","heartbeat_front_addr":"172.21.15.129:6818/10196","state":["exists","up"]},{"osd":1,"uuid":"cc8d989c-3ca6-4568-8bf3-c4550e51e8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":10197},{"type":"v1","addr":"172.21.15.129:6813","nonce":10197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6817","nonce":10197},{"type":"v1","addr":"172.21.15.129:6819","nonce":10197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":10197},{"type":"v1","addr":"172.21.15.129:6825","nonce":10197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":10197},{"type":"v1","addr":"172.21.15.129:6822","nonce":10197}]},"public_addr":"172.21.15.129:6813/10197","cluster_addr":"172.21.15.129:6819/10197","heartbeat_back_addr":"172.21.15.129:6825/10197","heartbeat_front_addr":"172.21.15.129:6822/10197","state":["exists","up"]},{"osd":2,"uuid":"708c0e93-0abe-42db-a152-1c035b1275d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":10195},{"type":"v1","addr":"172.21.15.129:6803","nonce":10195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":10195},{"type":"v1","addr":"172.21.15.129:6805","nonce":10195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":10195},{"type":"v1","addr":"172.21.15.129:6809","nonce":10195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":10195},{"type":"v1","addr":"172.21.15.129:6807","nonce":10195}]},"public_addr":"172.21.15.129:6803/10195","cluster_addr":"172.21.15.129:6805/10195","heartbeat_back_addr":"172.21.15.129:6809/10195","heartbeat_front_addr":"172.21.15.129:6807/10195","state":["exists","up"]},{"osd":3,"uuid":"fdc80253-76ed-4781-b47b-c152c1226e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":10198},{"type":"v1","addr":"172.21.15.129:6827","nonce":10198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":10198},{"type":"v1","addr":"172.21.15.129:6829","nonce":10198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":10198},{"type":"v1","addr":"172.21.15.129:6833","nonce":10198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":10198},{"type":"v1","addr":"172.21.15.129:6831","nonce":10198}]},"public_addr":"172.21.15.129:6827/10198","cluster_addr":"172.21.15.129:6829/10198","heartbeat_back_addr":"172.21.15.129:6833/10198","heartbeat_front_addr":"172.21.15.129:6831/10198","state":["exists","up"]},{"osd":4,"uuid":"0161ca95-e9d5-4ccf-86fc-5bedb6d2fcc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":10373},{"type":"v1","addr":"172.21.15.181:6801","nonce":10373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":10373},{"type":"v1","addr":"172.21.15.181:6805","nonce":10373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":10373},{"type":"v1","addr":"172.21.15.181:6820","nonce":10373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":10373},{"type":"v1","addr":"172.21.15.181:6812","nonce":10373}]},"public_addr":"172.21.15.181:6801/10373","cluster_addr":"172.21.15.181:6805/10373","heartbeat_back_addr":"172.21.15.181:6820/10373","heartbeat_front_addr":"172.21.15.181:6812/10373","state":["exists","up"]},{"osd":5,"uuid":"c9455e5c-2b38-4d12-9940-3d3e62337493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":10376},{"type":"v1","addr":"172.21.15.181:6803","nonce":10376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":10376},{"type":"v1","addr":"172.21.15.181:6808","nonce":10376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":10376},{"type":"v1","addr":"172.21.15.181:6818","nonce":10376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":10376},{"type":"v1","addr":"172.21.15.181:6813","nonce":10376}]},"public_addr":"172.21.15.181:6803/10376","cluster_addr":"172.21.15.181:6808/10376","heartbeat_back_addr":"172.21.15.181:6818/10376","heartbeat_front_addr":"172.21.15.181:6813/10376","state":["exists","up"]},{"osd":6,"uuid":"2ca613fd-fbe4-454a-8b48-a40e0ffd7e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":10374},{"type":"v1","addr":"172.21.15.181:6825","nonce":10374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":10374},{"type":"v1","addr":"172.21.15.181:6827","nonce":10374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":10374},{"type":"v1","addr":"172.21.15.181:6831","nonce":10374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":10374},{"type":"v1","addr":"172.21.15.181:6829","nonce":10374}]},"public_addr":"172.21.15.181:6825/10374","cluster_addr":"172.21.15.181:6827/10374","heartbeat_back_addr":"172.21.15.181:6831/10374","heartbeat_front_addr":"172.21.15.181:6829/10374","state":["exists","up"]},{"osd":7,"uuid":"fea090fa-cc6a-460a-8b78-ff8111552350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":10375},{"type":"v1","addr":"172.21.15.181:6809","nonce":10375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":10375},{"type":"v1","addr":"172.21.15.181:6817","nonce":10375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":10375},{"type":"v1","addr":"172.21.15.181:6824","nonce":10375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":10375},{"type":"v1","addr":"172.21.15.181:6821","nonce":10375}]},"public_addr":"172.21.15.181:6809/10375","cluster_addr":"172.21.15.181:6817/10375","heartbeat_back_addr":"172.21.15.181:6824/10375","heartbeat_front_addr":"172.21.15.181:6821/10375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.409855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.091424+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.567144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.187006+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.097427+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:09.114663+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.989105+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-08T12:44:08.848599+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-08T12:44:22.177 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-12-08T12:44:22.566 INFO:teuthology.orchestra.run.smithi129.stdout:55834574853 2019-12-08T12:44:22.567 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-12-08T12:44:23.255 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2019-12-08T12:44:23.256 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-12-08T12:44:23.729 INFO:teuthology.orchestra.run.smithi129.stdout:60129542149 2019-12-08T12:44:23.729 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-12-08T12:44:24.202 INFO:teuthology.orchestra.run.smithi129.stdout:55834574853 2019-12-08T12:44:24.202 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-12-08T12:44:24.751 INFO:teuthology.orchestra.run.smithi129.stdout:60129542148 2019-12-08T12:44:24.752 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-12-08T12:44:25.361 INFO:teuthology.orchestra.run.smithi129.stdout:55834574852 2019-12-08T12:44:25.361 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-12-08T12:44:25.966 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2019-12-08T12:44:25.966 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-12-08T12:44:26.528 INFO:teuthology.orchestra.run.smithi129.stdout:55834574852 2019-12-08T12:44:26.528 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-12-08T12:44:27.246 INFO:teuthology.orchestra.run.smithi129.stdout:55834574853 2019-12-08T12:44:27.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2019-12-08T12:44:27.261 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-12-08T12:44:27.998 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2019-12-08T12:44:28.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2019-12-08T12:44:28.014 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-12-08T12:44:28.709 INFO:teuthology.orchestra.run.smithi129.stdout:60129542150 2019-12-08T12:44:28.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.2 2019-12-08T12:44:28.723 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-12-08T12:44:29.415 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2019-12-08T12:44:29.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.3 2019-12-08T12:44:29.436 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-12-08T12:44:30.318 INFO:teuthology.orchestra.run.smithi129.stdout:60129542149 2019-12-08T12:44:30.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.4 2019-12-08T12:44:30.339 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-12-08T12:44:31.148 INFO:teuthology.orchestra.run.smithi129.stdout:55834574853 2019-12-08T12:44:31.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.5 2019-12-08T12:44:31.163 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-12-08T12:44:31.900 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2019-12-08T12:44:31.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2019-12-08T12:44:31.913 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-12-08T12:44:32.660 INFO:teuthology.orchestra.run.smithi129.stdout:55834574853 2019-12-08T12:44:32.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2019-12-08T12:44:32.679 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-12-08T12:44:32.679 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-08T12:44:33.387 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2019-12-08T12:44:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:33.399 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-12-08T12:44:32.335262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390240,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584480,"statfs":{"total":773094113280,"available":764502507520,"internally_reserved":8589934592,"allocated":1605632,"data_stored":522376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.333864+0000","last_change":"2019-12-08T12:44:18.554972+0000","last_active":"2019-12-08T12:44:19.333864+0000","last_peered":"2019-12-08T12:44:19.333864+0000","last_clean":"2019-12-08T12:44:19.333864+0000","last_became_active":"2019-12-08T12:44:18.529261+0000","last_became_peered":"2019-12-08T12:44:18.529261+0000","last_unstale":"2019-12-08T12:44:19.333864+0000","last_undegraded":"2019-12-08T12:44:19.333864+0000","last_fullsized":"2019-12-08T12:44:19.333864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.338360+0000","last_change":"2019-12-08T12:44:18.875768+0000","last_active":"2019-12-08T12:44:19.338360+0000","last_peered":"2019-12-08T12:44:19.338360+0000","last_clean":"2019-12-08T12:44:19.338360+0000","last_became_active":"2019-12-08T12:44:18.857009+0000","last_became_peered":"2019-12-08T12:44:18.857009+0000","last_unstale":"2019-12-08T12:44:19.338360+0000","last_undegraded":"2019-12-08T12:44:19.338360+0000","last_fullsized":"2019-12-08T12:44:19.338360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.339778+0000","last_change":"2019-12-08T12:44:19.136509+0000","last_active":"2019-12-08T12:44:19.339778+0000","last_peered":"2019-12-08T12:44:19.339778+0000","last_clean":"2019-12-08T12:44:19.339778+0000","last_became_active":"2019-12-08T12:44:19.091174+0000","last_became_peered":"2019-12-08T12:44:19.091174+0000","last_unstale":"2019-12-08T12:44:19.339778+0000","last_undegraded":"2019-12-08T12:44:19.339778+0000","last_fullsized":"2019-12-08T12:44:19.339778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.637496+0000","last_change":"2019-12-08T12:44:18.641933+0000","last_active":"2019-12-08T12:44:19.637496+0000","last_peered":"2019-12-08T12:44:19.637496+0000","last_clean":"2019-12-08T12:44:19.637496+0000","last_became_active":"2019-12-08T12:44:18.599275+0000","last_became_peered":"2019-12-08T12:44:18.599275+0000","last_unstale":"2019-12-08T12:44:19.637496+0000","last_undegraded":"2019-12-08T12:44:19.637496+0000","last_fullsized":"2019-12-08T12:44:19.637496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.788938+0000","last_change":"2019-12-08T12:44:19.751285+0000","last_active":"2019-12-08T12:44:19.788938+0000","last_peered":"2019-12-08T12:44:19.788938+0000","last_clean":"2019-12-08T12:44:19.788938+0000","last_became_active":"2019-12-08T12:44:19.702191+0000","last_became_peered":"2019-12-08T12:44:19.702191+0000","last_unstale":"2019-12-08T12:44:19.788938+0000","last_undegraded":"2019-12-08T12:44:19.788938+0000","last_fullsized":"2019-12-08T12:44:19.788938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.763186+0000","last_change":"2019-12-08T12:44:19.744013+0000","last_active":"2019-12-08T12:44:19.763186+0000","last_peered":"2019-12-08T12:44:19.763186+0000","last_clean":"2019-12-08T12:44:19.763186+0000","last_became_active":"2019-12-08T12:44:19.696793+0000","last_became_peered":"2019-12-08T12:44:19.696793+0000","last_unstale":"2019-12-08T12:44:19.763186+0000","last_undegraded":"2019-12-08T12:44:19.763186+0000","last_fullsized":"2019-12-08T12:44:19.763186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.791160+0000","last_change":"2019-12-08T12:44:19.770499+0000","last_active":"2019-12-08T12:44:19.791160+0000","last_peered":"2019-12-08T12:44:19.791160+0000","last_clean":"2019-12-08T12:44:19.791160+0000","last_became_active":"2019-12-08T12:44:19.724261+0000","last_became_peered":"2019-12-08T12:44:19.724261+0000","last_unstale":"2019-12-08T12:44:19.791160+0000","last_undegraded":"2019-12-08T12:44:19.791160+0000","last_fullsized":"2019-12-08T12:44:19.791160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.654743+0000","last_change":"2019-12-08T12:44:18.339414+0000","last_active":"2019-12-08T12:44:19.654743+0000","last_peered":"2019-12-08T12:44:19.654743+0000","last_clean":"2019-12-08T12:44:19.654743+0000","last_became_active":"2019-12-08T12:44:18.305194+0000","last_became_peered":"2019-12-08T12:44:18.305194+0000","last_unstale":"2019-12-08T12:44:19.654743+0000","last_undegraded":"2019-12-08T12:44:19.654743+0000","last_fullsized":"2019-12-08T12:44:19.654743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.112}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.535}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.390}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.887}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":59.347}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.342}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":44.521}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.339}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.328}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.205}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.612}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.164}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.789}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.939}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.482}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.463}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.987}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.492}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.243}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.054}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.705}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.770}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.298}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.253}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.129}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.031}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.958}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.408}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.032}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.272}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.074}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.705}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.916}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.952}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.894}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.298}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.209}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.272}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.283}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.148}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.776}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.640}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.465}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.794}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":6.959}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.597}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.657}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.359}]}]}]}} 2019-12-08T12:44:33.402 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-08T12:44:34.122 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2019-12-08T12:44:34.122 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:34.141 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-12-08T12:44:32.335262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390240,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584480,"statfs":{"total":773094113280,"available":764502507520,"internally_reserved":8589934592,"allocated":1605632,"data_stored":522376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.333864+0000","last_change":"2019-12-08T12:44:18.554972+0000","last_active":"2019-12-08T12:44:19.333864+0000","last_peered":"2019-12-08T12:44:19.333864+0000","last_clean":"2019-12-08T12:44:19.333864+0000","last_became_active":"2019-12-08T12:44:18.529261+0000","last_became_peered":"2019-12-08T12:44:18.529261+0000","last_unstale":"2019-12-08T12:44:19.333864+0000","last_undegraded":"2019-12-08T12:44:19.333864+0000","last_fullsized":"2019-12-08T12:44:19.333864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.338360+0000","last_change":"2019-12-08T12:44:18.875768+0000","last_active":"2019-12-08T12:44:19.338360+0000","last_peered":"2019-12-08T12:44:19.338360+0000","last_clean":"2019-12-08T12:44:19.338360+0000","last_became_active":"2019-12-08T12:44:18.857009+0000","last_became_peered":"2019-12-08T12:44:18.857009+0000","last_unstale":"2019-12-08T12:44:19.338360+0000","last_undegraded":"2019-12-08T12:44:19.338360+0000","last_fullsized":"2019-12-08T12:44:19.338360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.339778+0000","last_change":"2019-12-08T12:44:19.136509+0000","last_active":"2019-12-08T12:44:19.339778+0000","last_peered":"2019-12-08T12:44:19.339778+0000","last_clean":"2019-12-08T12:44:19.339778+0000","last_became_active":"2019-12-08T12:44:19.091174+0000","last_became_peered":"2019-12-08T12:44:19.091174+0000","last_unstale":"2019-12-08T12:44:19.339778+0000","last_undegraded":"2019-12-08T12:44:19.339778+0000","last_fullsized":"2019-12-08T12:44:19.339778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.637496+0000","last_change":"2019-12-08T12:44:18.641933+0000","last_active":"2019-12-08T12:44:19.637496+0000","last_peered":"2019-12-08T12:44:19.637496+0000","last_clean":"2019-12-08T12:44:19.637496+0000","last_became_active":"2019-12-08T12:44:18.599275+0000","last_became_peered":"2019-12-08T12:44:18.599275+0000","last_unstale":"2019-12-08T12:44:19.637496+0000","last_undegraded":"2019-12-08T12:44:19.637496+0000","last_fullsized":"2019-12-08T12:44:19.637496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.788938+0000","last_change":"2019-12-08T12:44:19.751285+0000","last_active":"2019-12-08T12:44:19.788938+0000","last_peered":"2019-12-08T12:44:19.788938+0000","last_clean":"2019-12-08T12:44:19.788938+0000","last_became_active":"2019-12-08T12:44:19.702191+0000","last_became_peered":"2019-12-08T12:44:19.702191+0000","last_unstale":"2019-12-08T12:44:19.788938+0000","last_undegraded":"2019-12-08T12:44:19.788938+0000","last_fullsized":"2019-12-08T12:44:19.788938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.763186+0000","last_change":"2019-12-08T12:44:19.744013+0000","last_active":"2019-12-08T12:44:19.763186+0000","last_peered":"2019-12-08T12:44:19.763186+0000","last_clean":"2019-12-08T12:44:19.763186+0000","last_became_active":"2019-12-08T12:44:19.696793+0000","last_became_peered":"2019-12-08T12:44:19.696793+0000","last_unstale":"2019-12-08T12:44:19.763186+0000","last_undegraded":"2019-12-08T12:44:19.763186+0000","last_fullsized":"2019-12-08T12:44:19.763186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.791160+0000","last_change":"2019-12-08T12:44:19.770499+0000","last_active":"2019-12-08T12:44:19.791160+0000","last_peered":"2019-12-08T12:44:19.791160+0000","last_clean":"2019-12-08T12:44:19.791160+0000","last_became_active":"2019-12-08T12:44:19.724261+0000","last_became_peered":"2019-12-08T12:44:19.724261+0000","last_unstale":"2019-12-08T12:44:19.791160+0000","last_undegraded":"2019-12-08T12:44:19.791160+0000","last_fullsized":"2019-12-08T12:44:19.791160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-08T12:44:19.654743+0000","last_change":"2019-12-08T12:44:18.339414+0000","last_active":"2019-12-08T12:44:19.654743+0000","last_peered":"2019-12-08T12:44:19.654743+0000","last_clean":"2019-12-08T12:44:19.654743+0000","last_became_active":"2019-12-08T12:44:18.305194+0000","last_became_peered":"2019-12-08T12:44:18.305194+0000","last_unstale":"2019-12-08T12:44:19.654743+0000","last_undegraded":"2019-12-08T12:44:19.654743+0000","last_fullsized":"2019-12-08T12:44:19.654743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-08T12:44:16.566148+0000","last_clean_scrub_stamp":"2019-12-08T12:44:16.566148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.112}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.535}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.390}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.887}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":59.347}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.342}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":44.521}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.339}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.328}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.205}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.612}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.164}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.789}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.939}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.482}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.463}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.987}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.492}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.243}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.054}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.705}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.770}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.298}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.253}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.129}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.031}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.958}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.408}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.032}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.272}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.074}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.705}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.916}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.952}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.894}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.298}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.209}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.272}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.283}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.148}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.776}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.640}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048780,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323060,"statfs":{"total":96636764160,"available":95562813440,"internally_reserved":1073741824,"allocated":200704,"data_stored":65297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.465}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.794}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":6.959}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.597}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.657}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.359}]}]}]}} 2019-12-08T12:44:34.143 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-12-08T12:44:34.143 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-12-08T12:44:34.143 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-12-08T12:44:34.144 INFO:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-12-08T12:44:34.879 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:44:34.892 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-12-08T12:44:34.893 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-12-08T12:44:34.893 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-12-08T12:44:35.100 DEBUG:teuthology.task:Applying overrides for task openssl_keys: {'rgw.client.1': {'ca': 'root', 'client': 'client.1', 'embed-key': True}, 'root': {'key-type': 'rsa:4096', 'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0', 'client.1']}, 'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}} 2019-12-08T12:44:35.101 INFO:teuthology.orchestra.run.smithi129:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-08T12:44:35.190 INFO:teuthology.orchestra.run.smithi129:> openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/ubuntu/cephtest/ca/root.key -days 365 -out /home/ubuntu/cephtest/ca/root.crt -subj /CN=teuthology 2019-12-08T12:44:35.285 INFO:teuthology.orchestra.run.smithi129.stderr:Generating a 4096 bit RSA private key 2019-12-08T12:44:35.765 INFO:teuthology.orchestra.run.smithi129.stderr:...................................................................................................................................++ 2019-12-08T12:44:35.815 INFO:teuthology.orchestra.run.smithi129.stderr:..............++ 2019-12-08T12:44:35.815 INFO:teuthology.orchestra.run.smithi129.stderr:writing new private key to '/home/ubuntu/cephtest/ca/root.key' 2019-12-08T12:44:35.816 INFO:teuthology.orchestra.run.smithi129.stderr:----- 2019-12-08T12:44:35.822 INFO:teuthology.orchestra.run.smithi129:> sudo cp /home/ubuntu/cephtest/ca/root.crt /usr/share/pki/ca-trust-source/anchors/root.crt 2019-12-08T12:44:35.897 INFO:teuthology.orchestra.run.smithi129:> sudo update-ca-trust 2019-12-08T12:44:36.522 INFO:teuthology.orchestra.run.smithi181:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-08T12:44:36.616 INFO:tasks.openssl_keys:copying from ubuntu@smithi129.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/root.crt to ubuntu@smithi181.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/root.crt... 2019-12-08T12:44:36.684 DEBUG:teuthology.orchestra.remote:smithi129:/home/ubuntu/cephtest/ca/root.crt is 2KB 2019-12-08T12:44:36.862 INFO:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/ca/root.crt /usr/share/pki/ca-trust-source/anchors/root.crt 2019-12-08T12:44:36.964 INFO:teuthology.orchestra.run.smithi181:> sudo update-ca-trust 2019-12-08T12:44:37.861 INFO:teuthology.orchestra.run.smithi181:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-08T12:44:37.922 INFO:teuthology.orchestra.run.smithi129:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.1.key -out /home/ubuntu/cephtest/ca/rgw.client.1.csr -subj /CN=smithi181.front.sepia.ceph.com 2019-12-08T12:44:38.008 INFO:teuthology.orchestra.run.smithi129.stderr:Generating a 2048 bit RSA private key 2019-12-08T12:44:38.068 INFO:teuthology.orchestra.run.smithi129.stderr:.....................................+++ 2019-12-08T12:44:38.147 INFO:teuthology.orchestra.run.smithi129.stderr:.........................................................................+++ 2019-12-08T12:44:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:writing new private key to '/home/ubuntu/cephtest/ca/rgw.client.1.key' 2019-12-08T12:44:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:----- 2019-12-08T12:44:38.150 INFO:teuthology.orchestra.run.smithi129:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.1.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.1.crt -days 365 -sha256 2019-12-08T12:44:38.204 INFO:teuthology.orchestra.run.smithi129.stderr:Signature ok 2019-12-08T12:44:38.204 INFO:teuthology.orchestra.run.smithi129.stderr:subject=/CN=smithi181.front.sepia.ceph.com 2019-12-08T12:44:38.204 INFO:teuthology.orchestra.run.smithi129.stderr:Getting CA Private Key 2019-12-08T12:44:38.211 INFO:teuthology.orchestra.run.smithi129:> rm /home/ubuntu/cephtest/ca/rgw.client.1.csr /home/ubuntu/cephtest/ca/root.srl 2019-12-08T12:44:38.257 INFO:teuthology.orchestra.run.smithi129:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.1.crt 2019-12-08T12:44:38.308 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/ca/rgw.client.1.crt: OK 2019-12-08T12:44:38.308 INFO:tasks.openssl_keys:copying from ubuntu@smithi129.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.crt to ubuntu@smithi181.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.crt... 2019-12-08T12:44:38.357 DEBUG:teuthology.orchestra.remote:smithi129:/home/ubuntu/cephtest/ca/rgw.client.1.crt is 1KB 2019-12-08T12:44:38.506 INFO:tasks.openssl_keys:copying from ubuntu@smithi129.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.key to ubuntu@smithi181.front.sepia.ceph.com:/home/ubuntu/cephtest/ca/rgw.client.1.key... 2019-12-08T12:44:38.561 DEBUG:teuthology.orchestra.remote:smithi129:/home/ubuntu/cephtest/ca/rgw.client.1.key is 2KB 2019-12-08T12:44:38.712 INFO:teuthology.orchestra.run.smithi129:> rm /home/ubuntu/cephtest/ca/rgw.client.1.crt /home/ubuntu/cephtest/ca/rgw.client.1.key 2019-12-08T12:44:38.782 INFO:teuthology.orchestra.run.smithi181:> openssl verify /home/ubuntu/cephtest/ca/rgw.client.1.crt 2019-12-08T12:44:38.906 INFO:teuthology.orchestra.run.smithi181.stdout:/home/ubuntu/cephtest/ca/rgw.client.1.crt: OK 2019-12-08T12:44:38.907 INFO:teuthology.orchestra.run.smithi181:> cat /home/ubuntu/cephtest/ca/rgw.client.1.key >> /home/ubuntu/cephtest/ca/rgw.client.1.crt 2019-12-08T12:44:38.989 INFO:teuthology.orchestra.run.smithi129:> mkdir -p /home/ubuntu/cephtest/ca 2019-12-08T12:44:39.081 INFO:teuthology.orchestra.run.smithi129:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.0.key -out /home/ubuntu/cephtest/ca/rgw.client.0.csr -subj /CN=smithi129.front.sepia.ceph.com 2019-12-08T12:44:39.158 INFO:teuthology.orchestra.run.smithi129.stderr:Generating a 2048 bit RSA private key 2019-12-08T12:44:39.194 INFO:teuthology.orchestra.run.smithi129.stderr:.......................+++ 2019-12-08T12:44:39.198 INFO:teuthology.orchestra.run.smithi129.stderr:...+++ 2019-12-08T12:44:39.199 INFO:teuthology.orchestra.run.smithi129.stderr:writing new private key to '/home/ubuntu/cephtest/ca/rgw.client.0.key' 2019-12-08T12:44:39.199 INFO:teuthology.orchestra.run.smithi129.stderr:----- 2019-12-08T12:44:39.201 INFO:teuthology.orchestra.run.smithi129:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.0.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.0.crt -days 365 -sha256 2019-12-08T12:44:39.288 INFO:teuthology.orchestra.run.smithi129.stderr:Signature ok 2019-12-08T12:44:39.288 INFO:teuthology.orchestra.run.smithi129.stderr:subject=/CN=smithi129.front.sepia.ceph.com 2019-12-08T12:44:39.288 INFO:teuthology.orchestra.run.smithi129.stderr:Getting CA Private Key 2019-12-08T12:44:39.298 INFO:teuthology.orchestra.run.smithi129:> rm /home/ubuntu/cephtest/ca/rgw.client.0.csr /home/ubuntu/cephtest/ca/root.srl 2019-12-08T12:44:39.347 INFO:teuthology.orchestra.run.smithi129:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.0.crt 2019-12-08T12:44:39.401 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/ca/rgw.client.0.crt: OK 2019-12-08T12:44:39.402 INFO:teuthology.orchestra.run.smithi129:> cat /home/ubuntu/cephtest/ca/rgw.client.0.key >> /home/ubuntu/cephtest/ca/rgw.client.0.crt 2019-12-08T12:44:39.447 INFO:teuthology.run_tasks:Running task rgw... 2019-12-08T12:44:39.462 DEBUG:tasks.rgw:config is {'client.1': {'ssl certificate': 'rgw.client.1'}, 'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-12-08T12:44:39.462 DEBUG:tasks.rgw:client list is ['client.0'] 2019-12-08T12:44:39.462 INFO:tasks.rgw:Creating data pools 2019-12-08T12:44:39.462 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-12-08T12:44:39.463 INFO:teuthology.orchestra.run.smithi129:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=2 m=1 --cluster ceph 2019-12-08T12:44:40.658 INFO:teuthology.orchestra.run.smithi129:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-12-08T12:44:43.057 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'default.rgw.buckets.data' created 2019-12-08T12:44:43.072 INFO:teuthology.orchestra.run.smithi129:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-12-08T12:44:44.475 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-12-08T12:44:44.490 INFO:teuthology.orchestra.run.smithi129:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-12-08T12:44:45.640 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'default.rgw.buckets.index' created 2019-12-08T12:44:45.653 INFO:teuthology.orchestra.run.smithi129:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-12-08T12:44:46.762 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-12-08T12:44:46.781 DEBUG:tasks.rgw:Pools created 2019-12-08T12:44:46.781 INFO:tasks.rgw:Configuring compression type = random 2019-12-08T12:44:46.782 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-08T12:44:46.782 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-08T12:44:46.782 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-08T12:44:46.929 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:46.932+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:49.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.366+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:49.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.366+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:49.462 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.466+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:49.462 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.466+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:49.467 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.470+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:49.467 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.470+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:49.475+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.493+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:50.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.496+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:50.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.496+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:50.499 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.502+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:50.499 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.502+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:50.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.507+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:50.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.507+0000 7ff427a49ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T12:44:50.504 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.507+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:50.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.511+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:50.508 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.511+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:50.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.515+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:50.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.515+0000 7ff427a49ec0 10 Creating default zonegroup 2019-12-08T12:44:50.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.515+0000 7ff427a49ec0 20 get_system_obj_state: rctx=0x7ffecb992f40 obj=:.pools.avail state=0x555a47a2f670 s->prefetch_data=0 2019-12-08T12:44:50.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.517+0000 7ff427a49ec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-12-08T12:44:50.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.651+0000 7ff427a49ec0 10 WARNING: store->list_zones() returned r=-2 2019-12-08T12:44:50.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.651+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:50.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:50.660+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:51.429 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:51.433+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:51.436 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:51.440+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:51.436 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:51.440+0000 7ff427a49ec0 10 could not read realm id: (2) No such file or directory 2019-12-08T12:44:51.436 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:51.440+0000 7ff427a49ec0 10 WARNING: failed to set zone as default, r=-22 2019-12-08T12:44:51.437 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:51.440+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:51.447 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:51.451+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:52.109 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.112+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:52.177 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.181+0000 7ff427a49ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-08T12:44:52.178 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.181+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:52.239 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.243+0000 7ff427a49ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-08T12:44:52.240 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.243+0000 7ff427a49ec0 20 zonegroup default 2019-12-08T12:44:52.240 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.243+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:52.248 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.251+0000 7ff427a49ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:44:52.248 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.251+0000 7ff427a49ec0 10 Cannot find current period zone using local zone 2019-12-08T12:44:52.248 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.251+0000 7ff427a49ec0 10 Using default name default 2019-12-08T12:44:52.248 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.251+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:52.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.271+0000 7ff415474700 0 -- 172.21.15.129:0/4138418586 >> [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] conn(0x555a47a25230 msgr2=0x555a47a27650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T12:44:52.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.272+0000 7ff427a49ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:44:52.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.272+0000 7ff427a49ec0 20 rados->read ofs=0 len=0 2019-12-08T12:44:52.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.302+0000 7ff427a49ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-08T12:44:52.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.302+0000 7ff427a49ec0 20 zone default found 2019-12-08T12:44:52.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.302+0000 7ff427a49ec0 20 started sync module instance, tier type = 2019-12-08T12:44:52.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:52.302+0000 7ff427a49ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T12:44:55.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:55.924+0000 7ff415474700 0 -- 172.21.15.129:0/4138418586 >> [v2:172.21.15.129:6802/10195,v1:172.21.15.129:6803/10195] conn(0x555a47a2fb00 msgr2=0x555a47a31f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T12:44:56.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=0 2019-12-08T12:44:56.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=1 2019-12-08T12:44:56.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=2 2019-12-08T12:44:56.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=3 2019-12-08T12:44:56.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=4 2019-12-08T12:44:56.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=5 2019-12-08T12:44:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.664+0000 7ff427a49ec0 20 add_watcher() i=6 2019-12-08T12:44:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.706+0000 7ff427a49ec0 20 add_watcher() i=7 2019-12-08T12:44:56.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.706+0000 7ff427a49ec0 2 all 8 watchers are set, enabling cache 2019-12-08T12:44:56.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:44:56.706+0000 7ff3e4fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T12:45:00.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:00.546+0000 7ff427a49ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-12-08T12:45:05.236 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:05.238+0000 7ff427a49ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-12-08T12:45:09.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.042+0000 7ff356ffd700 20 reqs_thread_entry: start 2019-12-08T12:45:09.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.042+0000 7ff354ff9700 20 reqs_thread_entry: start 2019-12-08T12:45:09.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.043+0000 7ff427a49ec0 20 init_complete bucket index max shards: 0 2019-12-08T12:45:09.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.043+0000 7ff33affd700 20 reqs_thread_entry: start 2019-12-08T12:45:09.538 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.541+0000 7ff427a49ec0 20 remove_watcher() i=0 2019-12-08T12:45:09.539 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.541+0000 7ff427a49ec0 2 removed watcher, disabling cache 2019-12-08T12:45:09.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.641+0000 7ff427a49ec0 20 remove_watcher() i=1 2019-12-08T12:45:09.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.701+0000 7ff427a49ec0 20 remove_watcher() i=2 2019-12-08T12:45:09.762 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.765+0000 7ff427a49ec0 20 remove_watcher() i=3 2019-12-08T12:45:09.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.853+0000 7ff427a49ec0 20 remove_watcher() i=4 2019-12-08T12:45:09.911 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:09.913+0000 7ff427a49ec0 20 remove_watcher() i=5 2019-12-08T12:45:10.000 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.003+0000 7ff427a49ec0 20 remove_watcher() i=6 2019-12-08T12:45:10.075 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.078+0000 7ff427a49ec0 20 remove_watcher() i=7 2019-12-08T12:45:10.087 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2019-12-08T12:45:10.087 DEBUG:tasks.util.rgw: json result: [] 2019-12-08T12:45:10.087 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-08T12:45:10.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-08T12:45:10.088 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-12-08T12:45:10.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.361+0000 7f65337fe700 20 reqs_thread_entry: start 2019-12-08T12:45:10.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.361+0000 7f65feb53ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.376+0000 7f65feb53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:10.374 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.377+0000 7f65feb53ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.392+0000 7f65feb53ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-08T12:45:10.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.392+0000 7f65feb53ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.395 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.397+0000 7f65feb53ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:10.395 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.397+0000 7f65feb53ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.409+0000 7f65feb53ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:10.407 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.409+0000 7f65feb53ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.431 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.434+0000 7f65feb53ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-08T12:45:10.547 INFO:teuthology.orchestra.run.smithi129.stdout:{"id":"9161132f-da48-40b4-a853-f341a1971d48","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-08T12:45:10.548 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'9161132f-da48-40b4-a853-f341a1971d48', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-08T12:45:10.548 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-08T12:45:10.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-08T12:45:10.549 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-08T12:45:10.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.670+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.690 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.693+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:10.690 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.693+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.701+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:10.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.701+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.706+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:10.704 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.706+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.714+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:10.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.937+0000 7f6360300ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T12:45:10.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.937+0000 7f6360300ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T12:45:10.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.937+0000 7f6360300ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T12:45:10.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.937+0000 7f6360300ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T12:45:10.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.937+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.948+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:10.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.948+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.960+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-08T12:45:10.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.960+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.967 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.970+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:10.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.970+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.991+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-08T12:45:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:10.991+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:10.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.001+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:10.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.001+0000 7f6360300ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T12:45:10.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.001+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.006+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:11.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.007+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.016 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.018+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:11.016 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.018+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.034 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.038+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-08T12:45:11.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.038+0000 7f6360300ec0 20 zonegroup default 2019-12-08T12:45:11.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.038+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.048+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:11.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.048+0000 7f6360300ec0 10 Cannot find current period zone using local zone 2019-12-08T12:45:11.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.048+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.059+0000 7f6360300ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:11.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.059+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.069+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:11.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.069+0000 7f6360300ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:11.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.078+0000 7f6360300ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-08T12:45:11.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.079+0000 7f6360300ec0 20 zone default found 2019-12-08T12:45:11.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.079+0000 7f6360300ec0 20 started sync module instance, tier type = 2019-12-08T12:45:11.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.079+0000 7f6360300ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T12:45:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=0 2019-12-08T12:45:11.631 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=1 2019-12-08T12:45:11.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=2 2019-12-08T12:45:11.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=3 2019-12-08T12:45:11.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=4 2019-12-08T12:45:11.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=5 2019-12-08T12:45:11.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.634+0000 7f6360300ec0 20 add_watcher() i=6 2019-12-08T12:45:11.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.681+0000 7f6360300ec0 20 add_watcher() i=7 2019-12-08T12:45:11.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.681+0000 7f6360300ec0 2 all 8 watchers are set, enabling cache 2019-12-08T12:45:11.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.681+0000 7f631dfdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T12:45:11.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.686+0000 7f634cd29700 0 -- 172.21.15.129:0/1482820975 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55bab0559670 msgr2=0x55bab055bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T12:45:11.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:11.740+0000 7f634dd2b700 0 -- 172.21.15.129:0/2803781164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6290002720 msgr2=0x7f6290004b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T12:45:14.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.374+0000 7f628ffff700 20 reqs_thread_entry: start 2019-12-08T12:45:14.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.374+0000 7f628dffb700 20 reqs_thread_entry: start 2019-12-08T12:45:14.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.374+0000 7f6360300ec0 20 init_complete bucket index max shards: 0 2019-12-08T12:45:14.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.374+0000 7f627bfff700 20 reqs_thread_entry: start 2019-12-08T12:45:14.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.601+0000 7f6360300ec0 20 remove_watcher() i=0 2019-12-08T12:45:14.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.601+0000 7f6360300ec0 2 removed watcher, disabling cache 2019-12-08T12:45:14.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.669+0000 7f6360300ec0 20 remove_watcher() i=1 2019-12-08T12:45:14.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.735+0000 7f6360300ec0 20 remove_watcher() i=2 2019-12-08T12:45:14.802 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.804+0000 7f6360300ec0 20 remove_watcher() i=3 2019-12-08T12:45:14.856 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.859+0000 7f6360300ec0 20 remove_watcher() i=4 2019-12-08T12:45:14.927 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.929+0000 7f6360300ec0 20 remove_watcher() i=5 2019-12-08T12:45:14.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:14.973+0000 7f6360300ec0 20 remove_watcher() i=6 2019-12-08T12:45:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.022+0000 7f6360300ec0 20 remove_watcher() i=7 2019-12-08T12:45:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:[] 2019-12-08T12:45:15.028 DEBUG:tasks.util.rgw: json result: [] 2019-12-08T12:45:15.029 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-12-08T12:45:15.029 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-08T12:45:15.029 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-08T12:45:15.029 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-12-08T12:45:15.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.183+0000 7fed46ffd700 20 reqs_thread_entry: start 2019-12-08T12:45:15.181 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.183+0000 7fee11930ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.192+0000 7fee11930ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:15.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.192+0000 7fee11930ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.199+0000 7fee11930ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:15.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.199+0000 7fee11930ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.222 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.226+0000 7fee11930ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-08T12:45:15.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.226+0000 7fee11930ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.233+0000 7fee11930ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-08T12:45:15.418 INFO:teuthology.orchestra.run.smithi129.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-12-08T12:45:15.419 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-08T12:45:15.419 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-08T12:45:15.419 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-08T12:45:15.420 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-12-08T12:45:15.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.686+0000 7ff6a17fa700 20 reqs_thread_entry: start 2019-12-08T12:45:15.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.687+0000 7ff7643a7ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.706+0000 7ff7643a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:15.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.706+0000 7ff7643a7ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.716+0000 7ff7643a7ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-08T12:45:15.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.716+0000 7ff7643a7ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.721+0000 7ff7643a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:15.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.721+0000 7ff7643a7ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.725 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.729+0000 7ff7643a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:15.725 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.729+0000 7ff7643a7ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:15.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:15.757+0000 7ff7643a7ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-08T12:45:15.846 INFO:teuthology.orchestra.run.smithi129.stdout:{"id":"9161132f-da48-40b4-a853-f341a1971d48","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-08T12:45:15.848 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'9161132f-da48-40b4-a853-f341a1971d48', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-08T12:45:15.848 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-12-08T12:45:15.848 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-08T12:45:15.848 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-08T12:45:15.849 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-12-08T12:45:16.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.110+0000 7f971a7fc700 20 reqs_thread_entry: start 2019-12-08T12:45:16.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.110+0000 7f97e5576ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.118 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.121+0000 7f97e5576ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:16.118 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.121+0000 7f97e5576ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.133 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.135+0000 7f97e5576ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:16.134 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.135+0000 7f97e5576ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.161+0000 7f97e5576ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-08T12:45:16.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.161+0000 7f97e5576ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.174+0000 7f97e5576ec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-08T12:45:16.261 INFO:teuthology.orchestra.run.smithi129.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-12-08T12:45:16.261 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-08T12:45:16.262 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-08T12:45:16.262 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-08T12:45:16.263 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-12-08T12:45:16.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.537+0000 7fbf92ffd700 20 reqs_thread_entry: start 2019-12-08T12:45:16.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.538+0000 7fc05d817ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.551+0000 7fc05d817ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:16.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.551+0000 7fc05d817ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.564+0000 7fc05d817ec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-08T12:45:16.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.564+0000 7fc05d817ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.568 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.572+0000 7fc05d817ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:45:16.568 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.572+0000 7fc05d817ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.583+0000 7fc05d817ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:45:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.583+0000 7fc05d817ec0 20 rados->read ofs=0 len=0 2019-12-08T12:45:16.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:45:16.610+0000 7fc05d817ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T12:45:16.673 INFO:teuthology.orchestra.run.smithi129.stdout:{"id":"9161132f-da48-40b4-a853-f341a1971d48","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-08T12:45:16.673 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'9161132f-da48-40b4-a853-f341a1971d48', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-08T12:45:16.673 INFO:tasks.rgw:Starting rgw... 2019-12-08T12:45:16.673 INFO:tasks.rgw:rgw client.0 config is {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-12-08T12:45:16.674 INFO:tasks.rgw:Using beast as radosgw frontend 2019-12-08T12:45:16.674 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-12-08T12:45:16.674 INFO:tasks.rgw.client.0:Restarting daemon 2019-12-08T12:45:16.674 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-12-08T12:45:16.677 INFO:tasks.rgw.client.0:Started 2019-12-08T12:45:16.677 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on https://smithi129.front.sepia.ceph.com:443/ 2019-12-08T12:45:16.677 INFO:teuthology.orchestra.run.smithi129:> curl https://smithi129.front.sepia.ceph.com:443/ 2019-12-08T12:45:16.739 INFO:teuthology.orchestra.run.smithi129.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-08T12:45:16.739 INFO:teuthology.orchestra.run.smithi129.stderr: Dload Upload Total Spent Left Speed 2019-12-08T12:45:16.743 INFO:teuthology.orchestra.run.smithi129.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi129.front.sepia.ceph.com:443; Connection refused 2019-12-08T12:45:16.743 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-08T12:45:17.744 INFO:teuthology.orchestra.run.smithi129:> curl https://smithi129.front.sepia.ceph.com:443/ 2019-12-08T12:45:17.823 INFO:teuthology.orchestra.run.smithi129.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-08T12:45:17.823 INFO:teuthology.orchestra.run.smithi129.stderr: Dload Upload Total Spent Left Speed 2019-12-08T12:45:17.827 INFO:teuthology.orchestra.run.smithi129.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi129.front.sepia.ceph.com:443; Connection refused 2019-12-08T12:45:17.828 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-08T12:45:19.828 INFO:teuthology.orchestra.run.smithi129:> curl https://smithi129.front.sepia.ceph.com:443/ 2019-12-08T12:45:19.904 INFO:teuthology.orchestra.run.smithi129.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-08T12:45:19.904 INFO:teuthology.orchestra.run.smithi129.stderr: Dload Upload Total Spent Left Speed 2019-12-08T12:45:19.908 INFO:teuthology.orchestra.run.smithi129.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi129.front.sepia.ceph.com:443; Connection refused 2019-12-08T12:45:19.908 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-08T12:45:23.909 INFO:teuthology.orchestra.run.smithi129:> curl https://smithi129.front.sepia.ceph.com:443/ 2019-12-08T12:45:23.973 INFO:teuthology.orchestra.run.smithi129.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-08T12:45:23.973 INFO:teuthology.orchestra.run.smithi129.stderr: Dload Upload Total Spent Left Speed 2019-12-08T12:45:23.977 INFO:teuthology.orchestra.run.smithi129.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi129.front.sepia.ceph.com:443; Connection refused 2019-12-08T12:45:23.977 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-08T12:45:31.979 INFO:teuthology.orchestra.run.smithi129:> curl https://smithi129.front.sepia.ceph.com:443/ 2019-12-08T12:45:32.077 INFO:teuthology.orchestra.run.smithi129.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-08T12:45:32.079 INFO:teuthology.orchestra.run.smithi129.stderr: Dload Upload Total Spent Left Speed 2019-12-08T12:45:33.742 INFO:teuthology.orchestra.run.smithi129.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 100 214 0 214 0 0 128 0 --:--:-- 0:00:01 --:--:-- 128 2019-12-08T12:45:33.744 INFO:teuthology.orchestra.run.smithi129.stdout:anonymous 2019-12-08T12:45:33.744 INFO:teuthology.run_tasks:Running task workunit... 2019-12-08T12:45:33.760 INFO:tasks.workunit:Pulling workunits from ref 1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:45:33.760 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-12-08T12:45:33.760 INFO:teuthology.orchestra.run.smithi129:> stat -- /home/ubuntu/cephtest/mnt.0 2019-12-08T12:45:33.827 INFO:teuthology.orchestra.run.smithi129.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-12-08T12:45:33.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-08T12:45:33.828 INFO:teuthology.orchestra.run.smithi129:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-12-08T12:45:33.886 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-12-08T12:45:33.886 INFO:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-12-08T12:45:33.935 INFO:tasks.workunit:timeout=3h 2019-12-08T12:45:33.935 INFO:tasks.workunit:cleanup=True 2019-12-08T12:45:33.936 INFO:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1351c819e1a9518cde4c8b12d645782009028360 2019-12-08T12:45:34.196 INFO:tasks.workunit.client.0.smithi129.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-12-08T12:46:37.974 INFO:tasks.workunit.client.0.smithi129.stderr:Note: checking out '1351c819e1a9518cde4c8b12d645782009028360'. 2019-12-08T12:46:37.974 INFO:tasks.workunit.client.0.smithi129.stderr: 2019-12-08T12:46:37.975 INFO:tasks.workunit.client.0.smithi129.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-12-08T12:46:37.975 INFO:tasks.workunit.client.0.smithi129.stderr:changes and commit them, and you can discard any commits you make in this 2019-12-08T12:46:37.975 INFO:tasks.workunit.client.0.smithi129.stderr:state without impacting any branches by performing another checkout. 2019-12-08T12:46:37.975 INFO:tasks.workunit.client.0.smithi129.stderr: 2019-12-08T12:46:37.975 INFO:tasks.workunit.client.0.smithi129.stderr:If you want to create a new branch to retain commits you create, you may 2019-12-08T12:46:37.975 INFO:tasks.workunit.client.0.smithi129.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-12-08T12:46:37.976 INFO:tasks.workunit.client.0.smithi129.stderr: 2019-12-08T12:46:37.976 INFO:tasks.workunit.client.0.smithi129.stderr: git checkout -b new_branch_name 2019-12-08T12:46:37.976 INFO:tasks.workunit.client.0.smithi129.stderr: 2019-12-08T12:46:37.976 INFO:tasks.workunit.client.0.smithi129.stderr:HEAD is now at 1351c81... rgw: Fix dynamic resharding not working for empty zonegroup in period 2019-12-08T12:46:37.977 INFO:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-12-08T12:46:38.036 INFO:tasks.workunit.client.0.smithi129.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-12-08T12:46:38.038 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-08T12:46:38.039 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-12-08T12:46:38.535 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-12-08T12:46:38.618 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-12-08T12:46:38.666 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-08T12:46:38.668 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-08T12:46:38.668 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-12-08T12:46:38.708 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-08T12:46:38.755 DEBUG:teuthology.orchestra.remote:smithi129:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-12-08T12:46:38.805 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-12-08T12:46:38.805 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-12-08T12:46:38.805 INFO:teuthology.orchestra.run.smithi129:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-12-08T12:46:38.865 INFO:tasks.workunit.client.0.smithi129.stdout:Running main() from gmock_main.cc 2019-12-08T12:46:38.865 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] Running 11 tests from 1 test suite. 2019-12-08T12:46:38.865 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment set-up. 2019-12-08T12:46:38.865 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 11 tests from ClsLock 2019-12-08T12:46:38.865 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-12-08T12:46:41.440 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:41.440 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: exclusive 2019-12-08T12:46:41.441 INFO:tasks.workunit.client.0.smithi129.stdout: tag: 2019-12-08T12:46:41.441 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:41.441 INFO:tasks.workunit.client.0.smithi129.stdout: client.4400 expiration=2019-12-08T12:48:41.323516+0000 addr=v1:172.21.15.129:0/4174231773 cookie= 2019-12-08T12:46:41.566 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:41.566 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: shared 2019-12-08T12:46:41.567 INFO:tasks.workunit.client.0.smithi129.stdout: tag: 2019-12-08T12:46:41.567 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:41.567 INFO:tasks.workunit.client.0.smithi129.stdout: client.4400 expiration=2019-12-08T12:48:41.523035+0000 addr=v1:172.21.15.129:0/4174231773 cookie= 2019-12-08T12:46:41.567 INFO:tasks.workunit.client.0.smithi129.stdout: client.4519 expiration=0.000000 addr=v1:172.21.15.129:0/2079479296 cookie= 2019-12-08T12:46:41.626 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:41.626 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: shared 2019-12-08T12:46:41.627 INFO:tasks.workunit.client.0.smithi129.stdout: tag: 2019-12-08T12:46:41.627 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:41.627 INFO:tasks.workunit.client.0.smithi129.stdout: client.4519 expiration=0.000000 addr=v1:172.21.15.129:0/2079479296 cookie= 2019-12-08T12:46:41.813 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestMultiLocking (2949 ms) 2019-12-08T12:46:41.813 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestMeta 2019-12-08T12:46:44.965 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:44.966 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: shared 2019-12-08T12:46:44.966 INFO:tasks.workunit.client.0.smithi129.stdout: tag: 2019-12-08T12:46:44.966 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:44.966 INFO:tasks.workunit.client.0.smithi129.stdout: client.4415 expiration=0.000000 addr=v1:172.21.15.129:0/3304047827 cookie= 2019-12-08T12:46:45.039 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:45.040 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: exclusive 2019-12-08T12:46:45.040 INFO:tasks.workunit.client.0.smithi129.stdout: tag: new_tag 2019-12-08T12:46:45.040 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:45.040 INFO:tasks.workunit.client.0.smithi129.stdout: client.4409 expiration=0.000000 addr=v1:172.21.15.129:0/4255008183 cookie= 2019-12-08T12:46:45.334 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestMeta (3521 ms) 2019-12-08T12:46:45.335 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestCookie 2019-12-08T12:46:48.568 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:48.569 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: exclusive 2019-12-08T12:46:48.569 INFO:tasks.workunit.client.0.smithi129.stdout: tag: 2019-12-08T12:46:48.569 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:48.702 INFO:tasks.workunit.client.0.smithi129.stdout:lock: mylock 2019-12-08T12:46:48.702 INFO:tasks.workunit.client.0.smithi129.stdout: lock_type: shared 2019-12-08T12:46:48.702 INFO:tasks.workunit.client.0.smithi129.stdout: tag: 2019-12-08T12:46:48.703 INFO:tasks.workunit.client.0.smithi129.stdout: lockers: 2019-12-08T12:46:48.703 INFO:tasks.workunit.client.0.smithi129.stdout: client.4418 expiration=0.000000 addr=v1:172.21.15.129:0/2901457978 cookie= 2019-12-08T12:46:48.703 INFO:tasks.workunit.client.0.smithi129.stdout: client.4418 expiration=0.000000 addr=v1:172.21.15.129:0/2901457978 cookie=new cookie 2019-12-08T12:46:48.984 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestCookie (3649 ms) 2019-12-08T12:46:48.984 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-12-08T12:46:52.426 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestMultipleLocks (3443 ms) 2019-12-08T12:46:52.427 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestLockDuration 2019-12-08T12:47:00.670 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestLockDuration (8244 ms) 2019-12-08T12:47:00.670 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-12-08T12:47:04.170 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestAssertLocked (3499 ms) 2019-12-08T12:47:04.171 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestSetCookie 2019-12-08T12:47:07.442 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestSetCookie (3273 ms) 2019-12-08T12:47:07.442 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestRenew 2019-12-08T12:47:29.692 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestRenew (22250 ms) 2019-12-08T12:47:29.692 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-12-08T12:47:37.674 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7982 ms) 2019-12-08T12:47:37.674 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-12-08T12:47:46.354 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8680 ms) 2019-12-08T12:47:46.354 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-12-08T12:47:54.340 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7987 ms) 2019-12-08T12:47:54.340 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 11 tests from ClsLock (75477 ms total) 2019-12-08T12:47:54.340 INFO:tasks.workunit.client.0.smithi129.stdout: 2019-12-08T12:47:54.341 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment tear-down 2019-12-08T12:47:54.341 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] 11 tests from 1 test suite ran. (75477 ms total) 2019-12-08T12:47:54.341 INFO:tasks.workunit.client.0.smithi129.stdout:[ PASSED ] 11 tests. 2019-12-08T12:47:54.342 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-08T12:47:54.343 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-08T12:47:54.450 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-12-08T12:47:54.450 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-12-08T12:47:54.450 INFO:teuthology.orchestra.run.smithi129:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-12-08T12:47:54.511 INFO:tasks.workunit.client.0.smithi129.stdout:Running main() from gmock_main.cc 2019-12-08T12:47:54.511 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] Running 4 tests from 1 test suite. 2019-12-08T12:47:54.511 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment set-up. 2019-12-08T12:47:54.511 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 4 tests from cls_log 2019-12-08T12:47:54.511 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-12-08T12:47:57.904 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_log.test_log_add_same_time (3394 ms) 2019-12-08T12:47:57.904 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-12-08T12:48:01.288 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_log.test_log_add_different_time (3384 ms) 2019-12-08T12:48:01.288 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_log.trim_by_time 2019-12-08T12:48:06.290 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_log.trim_by_time (5002 ms) 2019-12-08T12:48:06.290 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_log.trim_by_marker 2019-12-08T12:48:10.700 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_log.trim_by_marker (4410 ms) 2019-12-08T12:48:10.700 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 4 tests from cls_log (16190 ms total) 2019-12-08T12:48:10.701 INFO:tasks.workunit.client.0.smithi129.stdout: 2019-12-08T12:48:10.701 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment tear-down 2019-12-08T12:48:10.701 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] 4 tests from 1 test suite ran. (16190 ms total) 2019-12-08T12:48:10.701 INFO:tasks.workunit.client.0.smithi129.stdout:[ PASSED ] 4 tests. 2019-12-08T12:48:10.702 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-08T12:48:10.702 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-08T12:48:10.770 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-12-08T12:48:10.770 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-12-08T12:48:10.771 INFO:teuthology.orchestra.run.smithi129:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-12-08T12:48:10.832 INFO:tasks.workunit.client.0.smithi129.stdout:Running main() from gmock_main.cc 2019-12-08T12:48:10.833 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] Running 5 tests from 1 test suite. 2019-12-08T12:48:10.833 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment set-up. 2019-12-08T12:48:10.833 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 5 tests from cls_rgw 2019-12-08T12:48:10.833 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.test_implicit 2019-12-08T12:48:14.172 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.test_implicit (3340 ms) 2019-12-08T12:48:14.173 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-12-08T12:48:17.853 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3680 ms) 2019-12-08T12:48:17.853 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.test_put_snap 2019-12-08T12:48:25.880 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T12:48:25.821+0000 37f93700 -1 osd.2 pg_epoch: 103 pg[25.6( v 102'4 (0'0,102'4] local-lis/les=99/100 n=2 ec=99/99 lis/c=99/99 les/c/f=100/100/0 sis=99) [2,1] r=0 lpr=99 crt=101'3 lcod 101'3 mlcod 101'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-12-08T12:48:27.824 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.test_put_snap (9971 ms) 2019-12-08T12:48:27.824 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.test_explicit 2019-12-08T12:48:31.067 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.test_explicit (3243 ms) 2019-12-08T12:48:31.067 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.set 2019-12-08T12:48:34.421 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.set (3352 ms) 2019-12-08T12:48:34.421 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 5 tests from cls_rgw (23587 ms total) 2019-12-08T12:48:34.422 INFO:tasks.workunit.client.0.smithi129.stdout: 2019-12-08T12:48:34.422 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment tear-down 2019-12-08T12:48:34.422 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] 5 tests from 1 test suite ran. (23588 ms total) 2019-12-08T12:48:34.422 INFO:tasks.workunit.client.0.smithi129.stdout:[ PASSED ] 5 tests. 2019-12-08T12:48:34.423 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-08T12:48:34.423 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-08T12:48:34.492 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-12-08T12:48:34.492 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-12-08T12:48:34.492 INFO:teuthology.orchestra.run.smithi129:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-12-08T12:48:34.571 INFO:tasks.workunit.client.0.smithi129.stdout:Running main() from gmock_main.cc 2019-12-08T12:48:34.571 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] Running 13 tests from 1 test suite. 2019-12-08T12:48:34.571 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment set-up. 2019-12-08T12:48:34.571 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 13 tests from cls_rgw 2019-12-08T12:48:36.708 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.index_basic 2019-12-08T12:48:38.406 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.index_basic (1697 ms) 2019-12-08T12:48:38.406 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-12-08T12:48:40.333 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (1927 ms) 2019-12-08T12:48:40.333 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.index_remove_object 2019-12-08T12:48:42.340 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.index_remove_object (2008 ms) 2019-12-08T12:48:42.341 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.index_suggest 2019-12-08T12:49:07.371 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.index_suggest (25031 ms) 2019-12-08T12:49:07.371 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.index_list 2019-12-08T12:49:09.128 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.index_list (1757 ms) 2019-12-08T12:49:09.128 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.bi_list 2019-12-08T12:49:12.248 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.bi_list (3120 ms) 2019-12-08T12:49:12.248 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.gc_set 2019-12-08T12:49:12.794 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.gc_set (546 ms) 2019-12-08T12:49:12.794 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.gc_list 2019-12-08T12:49:13.470 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.gc_list (676 ms) 2019-12-08T12:49:13.470 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.gc_defer 2019-12-08T12:49:20.806 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.gc_defer (7336 ms) 2019-12-08T12:49:20.806 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.usage_basic 2019-12-08T12:49:29.876 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.usage_basic (9070 ms) 2019-12-08T12:49:29.876 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-12-08T12:49:29.935 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.usage_clear_no_obj (59 ms) 2019-12-08T12:49:29.935 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.usage_clear 2019-12-08T12:49:40.266 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.usage_clear (10331 ms) 2019-12-08T12:49:40.266 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-12-08T12:49:42.209 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] cls_rgw.bi_log_trim (1944 ms) 2019-12-08T12:49:42.362 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 13 tests from cls_rgw (65502 ms total) 2019-12-08T12:49:42.362 INFO:tasks.workunit.client.0.smithi129.stdout: 2019-12-08T12:49:42.362 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment tear-down 2019-12-08T12:49:42.362 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] 13 tests from 1 test suite ran. (67792 ms total) 2019-12-08T12:49:42.362 INFO:tasks.workunit.client.0.smithi129.stdout:[ PASSED ] 13 tests. 2019-12-08T12:49:42.364 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-08T12:49:42.364 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-08T12:49:42.431 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-12-08T12:49:42.431 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-12-08T12:49:42.432 INFO:teuthology.orchestra.run.smithi129:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-12-08T12:49:42.497 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] Running 6 tests from 1 test suite. 2019-12-08T12:49:42.497 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment set-up. 2019-12-08T12:49:44.557 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 6 tests from rgw_gc_log 2019-12-08T12:49:44.558 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-12-08T12:49:45.159 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] rgw_gc_log.init_existing_queue (601 ms) 2019-12-08T12:49:45.159 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-12-08T12:49:45.253 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] rgw_gc_log.init_existing_omap (94 ms) 2019-12-08T12:49:45.253 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-12-08T12:49:45.344 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (91 ms) 2019-12-08T12:49:45.344 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-12-08T12:49:45.433 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (88 ms) 2019-12-08T12:49:45.433 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-12-08T12:49:45.540 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] rgw_gc_log.defer1_after_init (108 ms) 2019-12-08T12:49:45.541 INFO:tasks.workunit.client.0.smithi129.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-12-08T12:49:45.602 INFO:tasks.workunit.client.0.smithi129.stdout:[ OK ] rgw_gc_log.defer2_before_init (61 ms) 2019-12-08T12:49:45.602 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] 6 tests from rgw_gc_log (1044 ms total) 2019-12-08T12:49:45.602 INFO:tasks.workunit.client.0.smithi129.stdout: 2019-12-08T12:49:45.602 INFO:tasks.workunit.client.0.smithi129.stdout:[----------] Global test environment tear-down 2019-12-08T12:49:45.744 INFO:tasks.workunit.client.0.smithi129.stdout:[==========] 6 tests from 1 test suite ran. (3247 ms total) 2019-12-08T12:49:45.745 INFO:tasks.workunit.client.0.smithi129.stdout:[ PASSED ] 6 tests. 2019-12-08T12:49:45.748 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-08T12:49:45.748 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-08T12:49:45.866 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-12-08T12:49:45.866 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-12-08T12:49:46.123 DEBUG:teuthology.parallel:result is None 2019-12-08T12:49:46.124 INFO:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-12-08T12:49:46.180 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-12-08T12:49:46.181 INFO:teuthology.orchestra.run.smithi129:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-12-08T12:49:46.231 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-12-08T12:49:46.231 INFO:teuthology.run_tasks:Running task ragweed... 2019-12-08T12:49:46.248 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-12-08T12:49:46.249 INFO:tasks.ragweed:Downloading ragweed... 2019-12-08T12:49:46.249 INFO:tasks.ragweed:Using branch 'wip-ofriedma-dynamic-disable' for ragweed 2019-12-08T12:49:46.249 INFO:teuthology.orchestra.run.smithi129:> git clone -b wip-ofriedma-dynamic-disable git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-08T12:49:46.301 INFO:teuthology.orchestra.run.smithi129.stderr:fatal: Remote branch wip-ofriedma-dynamic-disable not found in upstream origin 2019-12-08T12:49:46.301 INFO:teuthology.orchestra.run.smithi129.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-08T12:49:46.301 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-12-08T12:49:46.302 INFO:teuthology.orchestra.run.smithi129:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-08T12:49:46.395 INFO:teuthology.orchestra.run.smithi129.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-08T12:49:46.491 INFO:tasks.ragweed:Creating rgw users... 2019-12-08T12:49:46.492 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-12-08T12:49:46.492 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key PNTJPCBDTVXXDZLSYNZJ --secret zDh0f2VTYzmTcOqZmI2kPpnjPMPxFcUHkMqu+LMyEwBEGkkLTfAaYQ== --email sysuser.client.0+test@test.test --system true 2019-12-08T12:49:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.614+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.641+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.641+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.649+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.649+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.646 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.654+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.646 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.654+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.658 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.666+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:46.884 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.891+0000 7efc043b2ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T12:49:46.884 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.891+0000 7efc043b2ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T12:49:46.884 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.891+0000 7efc043b2ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T12:49:46.884 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.891+0000 7efc043b2ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T12:49:46.884 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.891+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.894 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.902+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:46.894 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.902+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.910+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T12:49:46.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.910+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.911 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.919+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:46.912 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.919+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.932+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T12:49:46.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.932+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.936+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.929 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.936+0000 7efc043b2ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T12:49:46.929 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.936+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.942+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.942+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.955+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:46.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.955+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.965+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T12:49:46.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.965+0000 7efc043b2ec0 20 zonegroup default 2019-12-08T12:49:46.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.965+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.963 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.971+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.963 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.971+0000 7efc043b2ec0 10 Cannot find current period zone using local zone 2019-12-08T12:49:46.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.971+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.976+0000 7efc043b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:46.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.976+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.976 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.984+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:46.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.984+0000 7efc043b2ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:46.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.992+0000 7efc043b2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T12:49:46.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.993+0000 7efc043b2ec0 20 zone default found 2019-12-08T12:49:46.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.993+0000 7efc043b2ec0 20 started sync module instance, tier type = 2019-12-08T12:49:46.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:46.993+0000 7efc043b2ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T12:49:47.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=0 2019-12-08T12:49:47.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=1 2019-12-08T12:49:47.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=2 2019-12-08T12:49:47.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=3 2019-12-08T12:49:47.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=4 2019-12-08T12:49:47.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=5 2019-12-08T12:49:47.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.327+0000 7efc043b2ec0 20 add_watcher() i=6 2019-12-08T12:49:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.359+0000 7efc043b2ec0 20 add_watcher() i=7 2019-12-08T12:49:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.359+0000 7efc043b2ec0 2 all 8 watchers are set, enabling cache 2019-12-08T12:49:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.359+0000 7efbc1fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T12:49:47.354 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.362+0000 7efbf0ddb700 0 -- 172.21.15.129:0/1583974047 >> v2:172.21.15.129:3300/0 conn(0x55f26b3faf10 msgr2=0x55f26b3fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T12:49:47.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:47.408+0000 7efbf15dc700 0 -- 172.21.15.129:0/4005464464 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7efb34022480 msgr2=0x7efb34024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T12:49:48.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.785+0000 7efb33fff700 20 reqs_thread_entry: start 2019-12-08T12:49:48.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.785+0000 7efb31ffb700 20 reqs_thread_entry: start 2019-12-08T12:49:48.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.785+0000 7efc043b2ec0 20 init_complete bucket index max shards: 0 2019-12-08T12:49:48.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.785+0000 7efb1ffff700 20 reqs_thread_entry: start 2019-12-08T12:49:48.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.785+0000 7efc043b2ec0 20 get_system_obj_state: rctx=0x7ffd5fe736d8 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x55f26b478c70 s->prefetch_data=0 2019-12-08T12:49:48.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.785+0000 7efc043b2ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-08T12:49:48.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.799+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-12-08T12:49:48.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.799+0000 7efc043b2ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-08T12:49:48.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.799+0000 7efc043b2ec0 20 get_system_obj_state: rctx=0x7ffd5fe736e8 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x55f26b47aa00 s->prefetch_data=0 2019-12-08T12:49:48.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.799+0000 7efc043b2ec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-12-08T12:49:48.799 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.807+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-12-08T12:49:48.799 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.807+0000 7efc043b2ec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-08T12:49:48.799 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.807+0000 7efc043b2ec0 20 get_system_obj_state: rctx=0x7ffd5fe736e8 obj=default.rgw.meta:users.keys:PNTJPCBDTVXXDZLSYNZJ state=0x55f26b47aa00 s->prefetch_data=0 2019-12-08T12:49:48.799 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.807+0000 7efc043b2ec0 10 cache get: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ : miss 2019-12-08T12:49:48.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.813+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ info.flags=0x0 2019-12-08T12:49:48.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.813+0000 7efc043b2ec0 10 adding default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ to cache LRU end 2019-12-08T12:49:48.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.813+0000 7efc043b2ec0 20 get_system_obj_state: rctx=0x7ffd5fe732f8 obj=default.rgw.meta:users.keys:PNTJPCBDTVXXDZLSYNZJ state=0x55f26b47aa00 s->prefetch_data=0 2019-12-08T12:49:48.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.813+0000 7efc043b2ec0 10 cache get: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ : type miss (requested=0x6, cached=0x0) 2019-12-08T12:49:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.820+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ info.flags=0x0 2019-12-08T12:49:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.820+0000 7efc043b2ec0 10 moving default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ to cache LRU end 2019-12-08T12:49:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.820+0000 7efc043b2ec0 20 get_system_obj_state: rctx=0x7ffd5fe73478 obj=default.rgw.meta:users.keys:PNTJPCBDTVXXDZLSYNZJ state=0x55f26b47aa00 s->prefetch_data=0 2019-12-08T12:49:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.820+0000 7efc043b2ec0 10 cache get: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ : type miss (requested=0x6, cached=0x0) 2019-12-08T12:49:48.821 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.829+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ info.flags=0x0 2019-12-08T12:49:48.821 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.829+0000 7efc043b2ec0 10 moving default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ to cache LRU end 2019-12-08T12:49:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.890+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-08T12:49:48.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.890+0000 7efc043b2ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-08T12:49:48.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.890+0000 7efc043b2ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-12-08T12:49:48.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.930+0000 7efbd27fc700 10 RGWWatcher::handle_notify() notify_id 523986010112 cookie 94499669661936 notifier 4499 bl.length()=637 2019-12-08T12:49:48.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.930+0000 7efbd27fc700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-08T12:49:48.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:48.930+0000 7efbd27fc700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-08T12:49:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.078+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-08T12:49:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.078+0000 7efc043b2ec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-08T12:49:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.078+0000 7efc043b2ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-12-08T12:49:49.096 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.104+0000 7efbd27fc700 10 RGWWatcher::handle_notify() notify_id 523986010112 cookie 94499669678768 notifier 4499 bl.length()=202 2019-12-08T12:49:49.096 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.104+0000 7efbd27fc700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-08T12:49:49.097 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.104+0000 7efbd27fc700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-08T12:49:49.523 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.531+0000 7efc043b2ec0 10 cache put: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ info.flags=0x7 2019-12-08T12:49:49.524 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.531+0000 7efc043b2ec0 10 moving default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ to cache LRU end 2019-12-08T12:49:49.524 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.531+0000 7efc043b2ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=190 2019-12-08T12:49:49.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.597+0000 7efbd27fc700 10 RGWWatcher::handle_notify() notify_id 523986010112 cookie 94499669655072 notifier 4499 bl.length()=190 2019-12-08T12:49:49.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.597+0000 7efbd27fc700 10 cache put: name=default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ info.flags=0x7 2019-12-08T12:49:49.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:49.597+0000 7efbd27fc700 10 moving default.rgw.meta+users.keys+PNTJPCBDTVXXDZLSYNZJ to cache LRU end 2019-12-08T12:49:50.561 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T12:49:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "sysuser.client.0", 2019-12-08T12:49:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. sysuser.client.0", 2019-12-08T12:49:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "sysuser.client.0+test@test.test", 2019-12-08T12:49:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T12:49:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [], 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [ 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "sysuser.client.0", 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "access_key": "PNTJPCBDTVXXDZLSYNZJ", 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "zDh0f2VTYzmTcOqZmI2kPpnjPMPxFcUHkMqu+LMyEwBEGkkLTfAaYQ==" 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T12:49:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [], 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "system": "true", 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T12:49:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T12:49:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T12:49:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T12:49:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T12:49:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T12:49:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T12:49:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T12:49:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T12:49:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T12:49:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T12:49:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T12:49:50.566 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T12:49:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:49:50.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.650+0000 7efc043b2ec0 20 remove_watcher() i=0 2019-12-08T12:49:50.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.650+0000 7efc043b2ec0 2 removed watcher, disabling cache 2019-12-08T12:49:50.690 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.698+0000 7efc043b2ec0 20 remove_watcher() i=1 2019-12-08T12:49:50.720 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.728+0000 7efc043b2ec0 20 remove_watcher() i=2 2019-12-08T12:49:50.785 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.792+0000 7efc043b2ec0 20 remove_watcher() i=3 2019-12-08T12:49:50.834 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.842+0000 7efc043b2ec0 20 remove_watcher() i=4 2019-12-08T12:49:50.878 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.886+0000 7efc043b2ec0 20 remove_watcher() i=5 2019-12-08T12:49:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.930+0000 7efc043b2ec0 20 remove_watcher() i=6 2019-12-08T12:49:50.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:50.979+0000 7efc043b2ec0 20 remove_watcher() i=7 2019-12-08T12:49:50.982 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-12-08T12:49:50.982 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key RVZOKJJOYTFJVAUIOGTN --secret j+VEKNKCszaAcVItKInzjNqq5Jc2np7FxYf9Fw9YgcflshMQKqTG8Q== --email ragweed.client.0+test@test.test --system false 2019-12-08T12:49:51.146 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.153+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.169 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.177+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.177+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.192+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.192+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.196+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.196+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.204+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:51.365 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.373+0000 7fa430b81ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T12:49:51.365 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.373+0000 7fa430b81ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T12:49:51.365 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.373+0000 7fa430b81ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T12:49:51.365 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.373+0000 7fa430b81ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T12:49:51.366 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.373+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.380+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:51.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.380+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.387+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T12:49:51.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.387+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.387 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.395+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:51.387 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.395+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.404+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T12:49:51.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.404+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.409+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.409+0000 7fa430b81ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T12:49:51.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.409+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.414+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.414+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.414 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.422+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:51.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.422+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.432+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T12:49:51.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.432+0000 7fa430b81ec0 20 zonegroup default 2019-12-08T12:49:51.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.432+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.438+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.438+0000 7fa430b81ec0 10 Cannot find current period zone using local zone 2019-12-08T12:49:51.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.438+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.443+0000 7fa430b81ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T12:49:51.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.443+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.451+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T12:49:51.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.451+0000 7fa430b81ec0 20 rados->read ofs=0 len=0 2019-12-08T12:49:51.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.459+0000 7fa430b81ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T12:49:51.452 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.459+0000 7fa430b81ec0 20 zone default found 2019-12-08T12:49:51.452 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.459+0000 7fa430b81ec0 20 started sync module instance, tier type = 2019-12-08T12:49:51.452 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.459+0000 7fa430b81ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T12:49:51.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=0 2019-12-08T12:49:51.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=1 2019-12-08T12:49:51.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=2 2019-12-08T12:49:51.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=3 2019-12-08T12:49:51.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=4 2019-12-08T12:49:51.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=5 2019-12-08T12:49:51.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.818+0000 7fa430b81ec0 20 add_watcher() i=6 2019-12-08T12:49:51.842 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.850+0000 7fa430b81ec0 20 add_watcher() i=7 2019-12-08T12:49:51.842 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.850+0000 7fa430b81ec0 2 all 8 watchers are set, enabling cache 2019-12-08T12:49:51.842 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.850+0000 7fa3ee7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T12:49:51.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.862+0000 7fa41ddab700 0 -- 172.21.15.129:0/252735242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb1f73fde0 msgr2=0x55fb1f742200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T12:49:51.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:51.890+0000 7fa41e5ac700 0 -- 172.21.15.129:0/966881011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb1f742dd0 msgr2=0x55fb1f7431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T12:49:53.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.167+0000 7fa364ff9700 20 reqs_thread_entry: start 2019-12-08T12:49:53.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.167+0000 7fa35e7fc700 20 reqs_thread_entry: start 2019-12-08T12:49:53.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.167+0000 7fa430b81ec0 20 init_complete bucket index max shards: 0 2019-12-08T12:49:53.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.168+0000 7fa35cff9700 20 reqs_thread_entry: start 2019-12-08T12:49:53.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.168+0000 7fa430b81ec0 20 get_system_obj_state: rctx=0x7ffc9704fc38 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x55fb1f7bdb50 s->prefetch_data=0 2019-12-08T12:49:53.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.168+0000 7fa430b81ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-08T12:49:53.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.174+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-12-08T12:49:53.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.174+0000 7fa430b81ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-08T12:49:53.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.174+0000 7fa430b81ec0 20 get_system_obj_state: rctx=0x7ffc9704fc48 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x55fb1f7bf8e0 s->prefetch_data=0 2019-12-08T12:49:53.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.174+0000 7fa430b81ec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-12-08T12:49:53.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.181+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-12-08T12:49:53.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.181+0000 7fa430b81ec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-08T12:49:53.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.182+0000 7fa430b81ec0 20 get_system_obj_state: rctx=0x7ffc9704fc48 obj=default.rgw.meta:users.keys:RVZOKJJOYTFJVAUIOGTN state=0x55fb1f7bf8e0 s->prefetch_data=0 2019-12-08T12:49:53.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.182+0000 7fa430b81ec0 10 cache get: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN : miss 2019-12-08T12:49:53.193 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.201+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN info.flags=0x0 2019-12-08T12:49:53.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.201+0000 7fa430b81ec0 10 adding default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN to cache LRU end 2019-12-08T12:49:53.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.201+0000 7fa430b81ec0 20 get_system_obj_state: rctx=0x7ffc9704f858 obj=default.rgw.meta:users.keys:RVZOKJJOYTFJVAUIOGTN state=0x55fb1f7bf8e0 s->prefetch_data=0 2019-12-08T12:49:53.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.201+0000 7fa430b81ec0 10 cache get: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN : type miss (requested=0x6, cached=0x0) 2019-12-08T12:49:53.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.213+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN info.flags=0x0 2019-12-08T12:49:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.213+0000 7fa430b81ec0 10 moving default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN to cache LRU end 2019-12-08T12:49:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.213+0000 7fa430b81ec0 20 get_system_obj_state: rctx=0x7ffc9704f9d8 obj=default.rgw.meta:users.keys:RVZOKJJOYTFJVAUIOGTN state=0x55fb1f7bf8e0 s->prefetch_data=0 2019-12-08T12:49:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.213+0000 7fa430b81ec0 10 cache get: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN : type miss (requested=0x6, cached=0x0) 2019-12-08T12:49:53.212 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.220+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN info.flags=0x0 2019-12-08T12:49:53.212 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.220+0000 7fa430b81ec0 10 moving default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN to cache LRU end 2019-12-08T12:49:53.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.265+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-08T12:49:53.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.265+0000 7fa430b81ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-08T12:49:53.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.265+0000 7fa430b81ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-12-08T12:49:53.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.280+0000 7fa3feffd700 10 RGWWatcher::handle_notify() notify_id 523986010113 cookie 94537052764336 notifier 4698 bl.length()=637 2019-12-08T12:49:53.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.280+0000 7fa3feffd700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-08T12:49:53.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.280+0000 7fa3feffd700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-08T12:49:53.707 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.715+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-08T12:49:53.707 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.715+0000 7fa430b81ec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-08T12:49:53.707 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.715+0000 7fa430b81ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-12-08T12:49:53.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.731+0000 7fa3feffd700 10 RGWWatcher::handle_notify() notify_id 523986010113 cookie 94537052746048 notifier 4698 bl.length()=202 2019-12-08T12:49:53.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.731+0000 7fa3feffd700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-08T12:49:53.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:53.731+0000 7fa3feffd700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-08T12:49:53.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.003+0000 7fa430b81ec0 10 cache put: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN info.flags=0x7 2019-12-08T12:49:53.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.003+0000 7fa430b81ec0 10 moving default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN to cache LRU end 2019-12-08T12:49:53.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.003+0000 7fa430b81ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=190 2019-12-08T12:49:54.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.049+0000 7fa3feffd700 10 RGWWatcher::handle_notify() notify_id 523986010112 cookie 94537052742816 notifier 4698 bl.length()=190 2019-12-08T12:49:54.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.049+0000 7fa3feffd700 10 cache put: name=default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN info.flags=0x7 2019-12-08T12:49:54.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.049+0000 7fa3feffd700 10 moving default.rgw.meta+users.keys+RVZOKJJOYTFJVAUIOGTN to cache LRU end 2019-12-08T12:49:54.589 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T12:49:54.590 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "ragweed.client.0", 2019-12-08T12:49:54.590 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. ragweed.client.0", 2019-12-08T12:49:54.590 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "ragweed.client.0+test@test.test", 2019-12-08T12:49:54.590 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T12:49:54.590 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [], 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [ 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "ragweed.client.0", 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: "access_key": "RVZOKJJOYTFJVAUIOGTN", 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "j+VEKNKCszaAcVItKInzjNqq5Jc2np7FxYf9Fw9YgcflshMQKqTG8Q==" 2019-12-08T12:49:54.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [], 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: "system": "true", 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T12:49:54.592 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T12:49:54.593 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T12:49:54.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T12:49:54.595 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T12:49:54.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T12:49:54.595 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T12:49:54.595 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T12:49:54.595 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:49:54.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.658+0000 7fa430b81ec0 20 remove_watcher() i=0 2019-12-08T12:49:54.651 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.658+0000 7fa430b81ec0 2 removed watcher, disabling cache 2019-12-08T12:49:54.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.708+0000 7fa430b81ec0 20 remove_watcher() i=1 2019-12-08T12:49:54.739 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.747+0000 7fa430b81ec0 20 remove_watcher() i=2 2019-12-08T12:49:54.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.814+0000 7fa430b81ec0 20 remove_watcher() i=3 2019-12-08T12:49:54.870 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.878+0000 7fa430b81ec0 20 remove_watcher() i=4 2019-12-08T12:49:54.929 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.937+0000 7fa430b81ec0 20 remove_watcher() i=5 2019-12-08T12:49:54.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:54.973+0000 7fa430b81ec0 20 remove_watcher() i=6 2019-12-08T12:49:55.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T12:49:55.025+0000 7fa430b81ec0 20 remove_watcher() i=7 2019-12-08T12:49:55.023 INFO:tasks.ragweed:Configuring ragweed... 2019-12-08T12:49:55.023 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-12-08T12:50:01.066 INFO:teuthology.orchestra.run.smithi129.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-08T12:50:01.209 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T12:50:01.248 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T12:50:01.249 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T12:50:01.250 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T12:50:01.250 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T12:50:01.250 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T12:50:01.920 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T12:50:02.263 INFO:teuthology.orchestra.run.smithi129.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-12-08T12:50:03.175 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T12:50:03.176 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:50:03.176 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-12-08T12:50:03.178 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-12-08T12:50:03.180 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-12-08T12:50:03.213 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-12-08T12:50:03.214 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-12-08T12:50:03.217 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-12-08T12:50:03.218 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:50:03.218 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-12-08T12:50:03.220 INFO:teuthology.orchestra.run.smithi129.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-12-08T12:50:03.223 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-12-08T12:50:03.223 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T12:50:03.223 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-12-08T12:50:03.613 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T12:50:03.807 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:03.808 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T12:50:03.809 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:03.809 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:50:03.809 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T12:50:03.809 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout:Installing for dependencies: 2019-12-08T12:50:03.810 INFO:teuthology.orchestra.run.smithi129.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages (+3 Dependent packages) 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:03.811 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 1.7 M 2019-12-08T12:50:03.812 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 12 M 2019-12-08T12:50:03.812 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T12:50:04.153 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2019-12-08T12:50:04.153 INFO:teuthology.orchestra.run.smithi129.stdout:Total 4.9 MB/s | 1.7 MB 00:00 2019-12-08T12:50:04.158 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T12:50:04.167 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T12:50:04.217 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T12:50:04.217 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T12:50:04.917 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-12-08T12:50:05.181 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-12-08T12:50:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-08T12:50:05.923 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-12-08T12:50:06.378 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-12-08T12:50:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-12-08T12:50:06.942 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-12-08T12:50:07.059 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-12-08T12:50:07.177 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-12-08T12:50:07.252 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-08T12:50:07.319 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-12-08T12:50:07.420 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-12-08T12:50:07.520 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-12-08T12:50:07.970 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-12-08T12:50:07.970 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout:Dependency Installed: 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-12-08T12:50:07.971 INFO:teuthology.orchestra.run.smithi129.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-12-08T12:50:07.972 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:07.972 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T12:50:08.200 INFO:teuthology.orchestra.run.smithi129.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-12-08T12:50:09.777 INFO:teuthology.orchestra.run.smithi129.stdout:Installing setuptools, pip, wheel...done. 2019-12-08T12:50:10.251 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting pip 2019-12-08T12:50:10.355 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-12-08T12:50:10.611 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: pip 2019-12-08T12:50:10.611 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: pip 9.0.1 2019-12-08T12:50:10.674 INFO:teuthology.orchestra.run.smithi129.stdout: Uninstalling pip-9.0.1: 2019-12-08T12:50:10.727 INFO:teuthology.orchestra.run.smithi129.stdout: Successfully uninstalled pip-9.0.1 2019-12-08T12:50:11.314 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed pip-19.3.1 2019-12-08T12:50:12.080 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T12:50:12.156 INFO:teuthology.orchestra.run.smithi129.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-12-08T12:50:13.041 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T12:50:13.257 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting nose 2019-12-08T12:50:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-12-08T12:50:13.437 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: nose 2019-12-08T12:50:13.519 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed nose-1.3.7 2019-12-08T12:50:14.001 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T12:50:14.614 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting setuptools==32.3.1 2019-12-08T12:50:14.658 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-12-08T12:50:14.823 INFO:teuthology.orchestra.run.smithi129.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-08T12:50:14.823 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: setuptools 2019-12-08T12:50:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: setuptools 28.8.0 2019-12-08T12:50:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: Uninstalling setuptools-28.8.0: 2019-12-08T12:50:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-08T12:50:14.981 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed setuptools-32.3.1 2019-12-08T12:50:15.519 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T12:50:15.836 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting PyYAML 2019-12-08T12:50:15.881 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB) 2019-12-08T12:50:16.484 INFO:teuthology.orchestra.run.smithi129.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-12-08T12:50:16.646 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-12-08T12:50:16.658 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-12-08T12:50:16.946 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting bunch>=1.0.0 2019-12-08T12:50:16.956 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-12-08T12:50:17.723 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting gevent>=1.0 2019-12-08T12:50:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-12-08T12:50:18.154 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting httplib2 2019-12-08T12:50:18.166 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-12-08T12:50:19.261 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting lxml 2019-12-08T12:50:19.277 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-12-08T12:50:19.758 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-08T12:50:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T12:50:19.819 INFO:teuthology.orchestra.run.smithi129.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-12-08T12:50:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for PyYAML (setup.py): started 2019-12-08T12:50:20.503 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-12-08T12:50:20.504 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for PyYAML: filename=PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl size=46299 sha256=e23451319f665e382836bfdd9bc44d7e2aa7ac4a8e5f5626df64fb4cdfcf63e0 2019-12-08T12:50:20.504 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67 2019-12-08T12:50:20.519 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for bunch (setup.py): started 2019-12-08T12:50:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-12-08T12:50:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=5ae0fb5ad7a00369654b1abff851febe5f34fecc3058a89c091e579a524f0a5a 2019-12-08T12:50:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-12-08T12:50:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for httplib2 (setup.py): started 2019-12-08T12:50:21.182 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-12-08T12:50:21.183 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=3ffe4125927b59f2562487de830d046dec97f86e977cb0acdffc532641eb584d 2019-12-08T12:50:21.183 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-12-08T12:50:21.190 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully built PyYAML bunch httplib2 2019-12-08T12:50:21.345 INFO:teuthology.orchestra.run.smithi129.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-08T12:50:21.345 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-12-08T12:50:21.345 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: PyYAML 3.10 2019-12-08T12:50:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-08T12:50:21.346 INFO:teuthology.orchestra.run.smithi129.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-12-08T12:50:22.588 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: lxml 3.2.1 2019-12-08T12:50:22.588 INFO:teuthology.orchestra.run.smithi129.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-08T12:50:22.589 INFO:teuthology.orchestra.run.smithi129.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-12-08T12:50:22.658 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.2 2019-12-08T12:50:23.040 INFO:teuthology.orchestra.run.smithi129.stdout:running develop 2019-12-08T12:50:23.107 INFO:teuthology.orchestra.run.smithi129.stdout:running egg_info 2019-12-08T12:50:23.107 INFO:teuthology.orchestra.run.smithi129.stdout:creating ragweed.egg-info 2019-12-08T12:50:23.107 INFO:teuthology.orchestra.run.smithi129.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-12-08T12:50:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:writing ragweed.egg-info/PKG-INFO 2019-12-08T12:50:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-12-08T12:50:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-12-08T12:50:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-08T12:50:23.108 INFO:teuthology.orchestra.run.smithi129.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-08T12:50:23.109 INFO:teuthology.orchestra.run.smithi129.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-08T12:50:23.110 INFO:teuthology.orchestra.run.smithi129.stdout:running build_ext 2019-12-08T12:50:23.110 INFO:teuthology.orchestra.run.smithi129.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-12-08T12:50:23.110 INFO:teuthology.orchestra.run.smithi129.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-12-08T12:50:23.111 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.111 INFO:teuthology.orchestra.run.smithi129.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-12-08T12:50:23.112 INFO:teuthology.orchestra.run.smithi129.stdout:Processing dependencies for ragweed==0.0.1 2019-12-08T12:50:23.116 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for isodate>=0.4.4 2019-12-08T12:50:23.116 INFO:teuthology.orchestra.run.smithi129.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-12-08T12:50:23.207 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-12-08T12:50:23.257 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: isodate 0.6.0 2019-12-08T12:50:23.257 INFO:teuthology.orchestra.run.smithi129.stdout:Processing isodate-0.6.0.tar.gz 2019-12-08T12:50:23.266 INFO:teuthology.orchestra.run.smithi129.stdout:Writing /tmp/easy_install-ABjxS5/isodate-0.6.0/setup.cfg 2019-12-08T12:50:23.266 INFO:teuthology.orchestra.run.smithi129.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ABjxS5/isodate-0.6.0/egg-dist-tmp-MqhGvP 2019-12-08T12:50:23.549 INFO:teuthology.orchestra.run.smithi129.stderr:zip_safe flag not set; analyzing archive contents... 2019-12-08T12:50:23.564 INFO:teuthology.orchestra.run.smithi129.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-08T12:50:23.566 INFO:teuthology.orchestra.run.smithi129.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-08T12:50:23.567 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.567 INFO:teuthology.orchestra.run.smithi129.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-12-08T12:50:23.592 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for gevent==1.4.0 2019-12-08T12:50:23.593 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: gevent 1.4.0 2019-12-08T12:50:23.593 INFO:teuthology.orchestra.run.smithi129.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-08T12:50:23.593 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.593 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-08T12:50:23.594 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for bunch==1.0.1 2019-12-08T12:50:23.594 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: bunch 1.0.1 2019-12-08T12:50:23.594 INFO:teuthology.orchestra.run.smithi129.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-08T12:50:23.595 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.595 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-08T12:50:23.596 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for PyYAML==5.2 2019-12-08T12:50:23.596 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: PyYAML 5.2 2019-12-08T12:50:23.596 INFO:teuthology.orchestra.run.smithi129.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-08T12:50:23.596 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.596 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-08T12:50:23.597 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for boto==2.49.0 2019-12-08T12:50:23.597 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: boto 2.49.0 2019-12-08T12:50:23.597 INFO:teuthology.orchestra.run.smithi129.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-08T12:50:23.597 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.597 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-08T12:50:23.598 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for six==1.9.0 2019-12-08T12:50:23.598 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: six 1.9.0 2019-12-08T12:50:23.598 INFO:teuthology.orchestra.run.smithi129.stdout:Adding six 1.9.0 to easy-install.pth file 2019-12-08T12:50:23.598 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.598 INFO:teuthology.orchestra.run.smithi129.stdout:Using /usr/lib/python2.7/site-packages 2019-12-08T12:50:23.605 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for greenlet==0.4.15 2019-12-08T12:50:23.605 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: greenlet 0.4.15 2019-12-08T12:50:23.606 INFO:teuthology.orchestra.run.smithi129.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-08T12:50:23.606 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T12:50:23.606 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-08T12:50:23.606 INFO:teuthology.orchestra.run.smithi129.stdout:Finished processing dependencies for ragweed==0.0.1 2019-12-08T12:50:23.625 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-12-08T12:50:23.680 INFO:tasks.ragweed:Configuring boto... 2019-12-08T12:50:23.680 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-08T12:50:23.763 INFO:teuthology.orchestra.run.smithi129:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-12-08T12:51:38.482 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-12-08T12:52:40.008 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:09:11.449 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.063 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:09:11.502 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.089 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:09:11.529 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.113 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:09:11.594 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.118 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:09:11.558 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.181 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:09:11.662 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.644 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:09:12.080 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.671 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:09:12.110 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:40.840 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:09:12.321 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:52:42.790 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-12-08T12:53:32.643 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-12-08T12:54:41.272 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-12-08T12:54:55.591 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:11:27.035 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.597 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:11:27.041 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.609 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:11:27.051 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.616 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:11:27.111 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.618 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:11:27.060 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.626 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:11:27.122 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.708 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:11:27.151 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.730 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:11:27.171 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T12:54:55.738 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:11:27.233 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T12:56:09.986 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-12-08T12:58:42.439 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-12-08T13:00:03.012 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:16:34.460 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.045 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:16:34.493 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.052 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:16:34.499 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.054 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:16:34.500 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.061 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:16:34.592 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.064 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:16:34.594 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.220 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:16:34.669 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.245 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:16:34.692 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:00:03.305 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:16:34.835 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.454 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:17:32.898 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.470 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:17:32.914 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.484 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:17:32.926 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.503 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:17:32.945 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.514 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:17:32.959 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.548 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:17:32.991 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.550 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:17:33.087 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.563 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:17:33.100 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:01.611 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:17:33.148 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:01:03.399 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-12-08T13:05:14.637 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:21:46.071 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.668 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:21:46.101 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.676 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:21:46.111 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.688 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:21:46.121 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.715 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:21:46.281 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.719 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:21:46.285 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.833 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:21:46.267 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.849 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:21:46.282 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:05:14.911 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:21:46.477 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:06:09.807 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-12-08T13:07:05.044 INFO:teuthology.orchestra.run.smithi129.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-12-08T13:07:05.044 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T13:07:05.045 INFO:teuthology.orchestra.run.smithi129.stderr:---------------------------------------------------------------------- 2019-12-08T13:07:05.045 INFO:teuthology.orchestra.run.smithi129.stderr:Ran 9 tests in 1000.833s 2019-12-08T13:07:05.045 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T13:07:05.045 INFO:teuthology.orchestra.run.smithi129.stderr:OK 2019-12-08T13:07:05.064 INFO:tasks.ragweed:Cleaning up boto... 2019-12-08T13:07:05.064 INFO:teuthology.orchestra.run.smithi129:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-08T13:07:05.134 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-12-08T13:07:05.247 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.249+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.269 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.271+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.269 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.271+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.279+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.279+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.286+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.286+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.295 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.297+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:05.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.471+0000 7f3dde5baec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:07:05.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.471+0000 7f3dde5baec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:07:05.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.471+0000 7f3dde5baec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:07:05.470 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.471+0000 7f3dde5baec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:07:05.470 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.471+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.481+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:05.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.481+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.488 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.490+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:05.488 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.490+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.500+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.500+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.519+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:05.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.519+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.525 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.527+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.525 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.527+0000 7f3dde5baec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:07:05.526 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.527+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.532+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.532+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.542+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:05.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.542+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.554 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.556+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:05.554 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.556+0000 7f3dde5baec0 20 zonegroup default 2019-12-08T13:07:05.554 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.556+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.564+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.564+0000 7f3dde5baec0 10 Cannot find current period zone using local zone 2019-12-08T13:07:05.563 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.564+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.567 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.570+0000 7f3dde5baec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:05.568 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.570+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.579+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:05.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.579+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:05.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.589+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:05.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.589+0000 7f3dde5baec0 20 zone default found 2019-12-08T13:07:05.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.589+0000 7f3dde5baec0 20 started sync module instance, tier type = 2019-12-08T13:07:05.588 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.589+0000 7f3dde5baec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:07:05.969 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.971+0000 7f3dde5baec0 20 add_watcher() i=0 2019-12-08T13:07:05.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.971+0000 7f3dde5baec0 20 add_watcher() i=1 2019-12-08T13:07:05.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.971+0000 7f3dde5baec0 20 add_watcher() i=2 2019-12-08T13:07:05.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.971+0000 7f3dde5baec0 20 add_watcher() i=3 2019-12-08T13:07:05.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.971+0000 7f3dde5baec0 20 add_watcher() i=4 2019-12-08T13:07:05.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:05.971+0000 7f3dde5baec0 20 add_watcher() i=5 2019-12-08T13:07:05.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:06.001+0000 7f3dde5baec0 20 add_watcher() i=6 2019-12-08T13:07:06.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:06.019+0000 7f3dcb7e4700 0 -- 172.21.15.129:0/4139711946 >> [v2:172.21.15.181:6806/10375,v1:172.21.15.181:6809/10375] conn(0x55bd214d9d70 msgr2=0x55bd214dc190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:07:06.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:06.019+0000 7f3dde5baec0 20 add_watcher() i=7 2019-12-08T13:07:06.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:06.019+0000 7f3dde5baec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:07:06.018 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:06.020+0000 7f3d9bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:07:06.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:06.053+0000 7f3dcb7e4700 0 -- 172.21.15.129:0/1239445971 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55bd214efa80 msgr2=0x55bd214f1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:07:07.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.650+0000 7f3d0a7fc700 20 reqs_thread_entry: start 2019-12-08T13:07:07.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.651+0000 7f3d03fff700 20 reqs_thread_entry: start 2019-12-08T13:07:07.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.651+0000 7f3dde5baec0 20 init_complete bucket index max shards: 0 2019-12-08T13:07:07.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.651+0000 7f3d027fc700 20 reqs_thread_entry: start 2019-12-08T13:07:07.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.651+0000 7f3dde5baec0 20 get_system_obj_state: rctx=0x7ffde89fdba8 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x55bd2156af30 s->prefetch_data=0 2019-12-08T13:07:07.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.651+0000 7f3dde5baec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-08T13:07:07.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.665+0000 7f3dde5baec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-12-08T13:07:07.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.665+0000 7f3dde5baec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-08T13:07:07.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.665+0000 7f3dde5baec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:07.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.665+0000 7f3dde5baec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:07.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.665+0000 7f3dde5baec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T13:07:07.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.665+0000 7f3dde5baec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:07.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.679+0000 7f3dde5baec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-08T13:07:07.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.679+0000 7f3dde5baec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-12-08T13:07:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.679+0000 7f3dde5baec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-08T13:07:07.688 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.690+0000 7f3dde5baec0 10 removing key index: PNTJPCBDTVXXDZLSYNZJ 2019-12-08T13:07:07.689 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.690+0000 7f3dde5baec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=170 2019-12-08T13:07:07.710 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:07.713+0000 7f3dc0ff9700 10 RGWWatcher::handle_notify() notify_id 562640715777 cookie 94270795851632 notifier 4520 bl.length()=170 2019-12-08T13:07:08.085 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.088+0000 7f3dde5baec0 10 removing email index: sysuser.client.0+test@test.test 2019-12-08T13:07:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.088+0000 7f3dde5baec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-12-08T13:07:08.100 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.102+0000 7f3dc0ff9700 10 RGWWatcher::handle_notify() notify_id 562640715778 cookie 94270795860768 notifier 4520 bl.length()=182 2019-12-08T13:07:08.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.520+0000 7f3dde5baec0 10 removing user buckets index 2019-12-08T13:07:08.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.520+0000 7f3dde5baec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-12-08T13:07:08.582 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.585+0000 7f3dc0ff9700 10 RGWWatcher::handle_notify() notify_id 562640715776 cookie 94270795861888 notifier 4520 bl.length()=173 2019-12-08T13:07:08.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.967+0000 7f3dde5baec0 10 removing user index: sysuser.client.0 2019-12-08T13:07:08.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.967+0000 7f3dde5baec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-12-08T13:07:08.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.967+0000 7f3dde5baec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-12-08T13:07:08.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:08.979+0000 7f3dc0ff9700 10 RGWWatcher::handle_notify() notify_id 562640715778 cookie 94270795852656 notifier 4520 bl.length()=165 2019-12-08T13:07:09.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.157+0000 7f3dde5baec0 20 remove_watcher() i=0 2019-12-08T13:07:09.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.157+0000 7f3dde5baec0 2 removed watcher, disabling cache 2019-12-08T13:07:09.183 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.186+0000 7f3dde5baec0 20 remove_watcher() i=1 2019-12-08T13:07:09.212 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.214+0000 7f3dde5baec0 20 remove_watcher() i=2 2019-12-08T13:07:09.241 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.243+0000 7f3dde5baec0 20 remove_watcher() i=3 2019-12-08T13:07:09.275 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.278+0000 7f3dde5baec0 20 remove_watcher() i=4 2019-12-08T13:07:09.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.319+0000 7f3dde5baec0 20 remove_watcher() i=5 2019-12-08T13:07:09.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.353+0000 7f3dde5baec0 20 remove_watcher() i=6 2019-12-08T13:07:09.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.398+0000 7f3dde5baec0 20 remove_watcher() i=7 2019-12-08T13:07:09.407 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-12-08T13:07:09.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.555+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.581 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.583+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.581 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.583+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.591+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.592+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.594 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.596+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.594 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.596+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.604 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.606+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.776+0000 7fa10cf1bec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:07:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.776+0000 7fa10cf1bec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:07:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.776+0000 7fa10cf1bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:07:09.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.776+0000 7fa10cf1bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:07:09.775 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.776+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.788+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:09.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.789+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.800+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:09.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.800+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.811+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:09.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.811+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.831+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:09.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.831+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.838+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.838+0000 7fa10cf1bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:07:09.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.838+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.845 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.847+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.846 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.847+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.864+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:09.862 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.864+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.881 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.883+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:09.881 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.883+0000 7fa10cf1bec0 20 zonegroup default 2019-12-08T13:07:09.881 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.883+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.888+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.888+0000 7fa10cf1bec0 10 Cannot find current period zone using local zone 2019-12-08T13:07:09.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.888+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.892+0000 7fa10cf1bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:09.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.892+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.905+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:09.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.905+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:09.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.917+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:09.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.917+0000 7fa10cf1bec0 20 zone default found 2019-12-08T13:07:09.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.917+0000 7fa10cf1bec0 20 started sync module instance, tier type = 2019-12-08T13:07:09.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:09.917+0000 7fa10cf1bec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:07:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.346+0000 7fa10cf1bec0 20 add_watcher() i=0 2019-12-08T13:07:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.346+0000 7fa10cf1bec0 20 add_watcher() i=1 2019-12-08T13:07:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.346+0000 7fa10cf1bec0 20 add_watcher() i=2 2019-12-08T13:07:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.346+0000 7fa10cf1bec0 20 add_watcher() i=3 2019-12-08T13:07:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.346+0000 7fa10cf1bec0 20 add_watcher() i=4 2019-12-08T13:07:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.346+0000 7fa10cf1bec0 20 add_watcher() i=5 2019-12-08T13:07:10.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.369+0000 7fa10cf1bec0 20 add_watcher() i=6 2019-12-08T13:07:10.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.386+0000 7fa10cf1bec0 20 add_watcher() i=7 2019-12-08T13:07:10.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.386+0000 7fa10cf1bec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:07:10.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.386+0000 7fa0ca7e4700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:07:10.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.388+0000 7fa0fa946700 0 -- 172.21.15.129:0/3771718655 >> v2:172.21.15.129:3300/0 conn(0x55d79cc4f6f0 msgr2=0x55d79cc4fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T13:07:10.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:10.438+0000 7fa0f9944700 0 -- 172.21.15.129:0/3259603343 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55d79cc5cd40 msgr2=0x55d79cc5f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:07:11.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.851+0000 7fa040ff9700 20 reqs_thread_entry: start 2019-12-08T13:07:11.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.851+0000 7fa03a7fc700 20 reqs_thread_entry: start 2019-12-08T13:07:11.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.851+0000 7fa10cf1bec0 20 init_complete bucket index max shards: 0 2019-12-08T13:07:11.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.851+0000 7fa038ff9700 20 reqs_thread_entry: start 2019-12-08T13:07:11.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.851+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aef38 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x55d79ccd2cb0 s->prefetch_data=0 2019-12-08T13:07:11.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.851+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-08T13:07:11.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.862+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-12-08T13:07:11.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.862+0000 7fa10cf1bec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-08T13:07:11.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.862+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:11.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.862+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:11.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.862+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T13:07:11.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.862+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:11.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.876+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-08T13:07:11.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.876+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-12-08T13:07:11.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.876+0000 7fa10cf1bec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-08T13:07:11.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.900+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-conf state=0x55d79ccd45d0 s->prefetch_data=0 2019-12-08T13:07:11.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.900+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-12-08T13:07:11.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.910+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-12-08T13:07:11.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.910+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-08T13:07:11.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.910+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:11.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.910+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:11.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.910+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:11.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.910+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:11.917 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.920+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=200 2019-12-08T13:07:11.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.920+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-12-08T13:07:11.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.920+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-08T13:07:11.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.920+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 state=0x55d79ccd6b90 s->prefetch_data=0 2019-12-08T13:07:11.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.920+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 : miss 2019-12-08T13:07:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 info.flags=0x16 2019-12-08T13:07:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 to cache LRU end 2019-12-08T13:07:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.935+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:11.952 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.955+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=292 2019-12-08T13:07:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.955+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 info.flags=0x13 2019-12-08T13:07:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.955+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 to cache LRU end 2019-12-08T13:07:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.955+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.955+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.955+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:11.969 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:11.971+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-conf[9161132f-da48-40b4-a853-f341a1971d48.4507.1]) start [] num_entries 1100 2019-12-08T13:07:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-12-08T13:07:12.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-12-08T13:07:12.007 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-12-08T13:07:12.007 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.007+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.023+0000 7fa10cf1bec0 10 manifest: total_size = 3912 2019-12-08T13:07:12.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.023+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.13 2019-12-08T13:07:12.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.023+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.023+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:12.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.023+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:12.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.229+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.258+0000 7fa10cf1bec0 10 manifest: total_size = 5842 2019-12-08T13:07:12.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.258+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.30 2019-12-08T13:07:12.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.258+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.258+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:12.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.258+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:12.482 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.484+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.509+0000 7fa10cf1bec0 10 manifest: total_size = 3778 2019-12-08T13:07:12.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.509+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.56 2019-12-08T13:07:12.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.509+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.509+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:12.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.509+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:12.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.703+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.712+0000 7fa10cf1bec0 10 manifest: total_size = 3826 2019-12-08T13:07:12.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.712+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.72 2019-12-08T13:07:12.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.712+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.712+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:12.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.712+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:12.891 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.892+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.910 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.911+0000 7fa10cf1bec0 10 manifest: total_size = 10477 2019-12-08T13:07:12.911 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.911+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.87 2019-12-08T13:07:12.911 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.911+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:12.911 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.911+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:12.911 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:12.911+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:13.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.066+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.077+0000 7fa10cf1bec0 10 manifest: total_size = 6037 2019-12-08T13:07:13.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.077+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.120 2019-12-08T13:07:13.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.077+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.077+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:13.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.077+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:13.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.204+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.212 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.213+0000 7fa10cf1bec0 10 manifest: total_size = 4115 2019-12-08T13:07:13.212 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.213+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.189 2019-12-08T13:07:13.213 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.213+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.213 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.213+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:13.213 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.213+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:13.347 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.348+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.369+0000 7fa10cf1bec0 10 manifest: total_size = 6115 2019-12-08T13:07:13.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.369+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.246 2019-12-08T13:07:13.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.369+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.369+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:13.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.369+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:13.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.544+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.581+0000 7fa10cf1bec0 10 manifest: total_size = 4010 2019-12-08T13:07:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.581+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.322 2019-12-08T13:07:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.581+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55d79ccdbd98 s->prefetch_data=0 2019-12-08T13:07:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.581+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd98 2019-12-08T13:07:13.581 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.581+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.1 2019-12-08T13:07:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.712+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-conf[9161132f-da48-40b4-a853-f341a1971d48.4507.1]) start _multipart_[] num_entries 1001 2019-12-08T13:07:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.712+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:13.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.775+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[9161132f-da48-40b4-a853-f341a1971d48.4507.1])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:13.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.856+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-conf state=0x55d79ccdce70 s->prefetch_data=0 2019-12-08T13:07:13.855 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.856+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:13.872 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-12-08T13:07:13.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-08T13:07:13.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:13.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:13.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:13.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-12-08T13:07:13.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.874+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-12-08T13:07:13.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:13.891+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715778 cookie 94384536390704 notifier 4678 bl.length()=162 2019-12-08T13:07:14.288 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.289+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:9161132f-da48-40b4-a853-f341a1971d48.4507.1 from cache 2019-12-08T13:07:14.288 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.289+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=219 2019-12-08T13:07:14.304 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.305+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715779 cookie 94384536416240 notifier 4678 bl.length()=219 2019-12-08T13:07:14.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.730+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x55d79ccd6b20 s->prefetch_data=0 2019-12-08T13:07:14.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.730+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-12-08T13:07:14.736 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.737+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-12-08T13:07:14.736 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.737+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-08T13:07:14.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.737+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:14.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.737+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:14.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.737+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:14.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.737+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:14.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.747+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=229 2019-12-08T13:07:14.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.747+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-12-08T13:07:14.747 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.747+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-08T13:07:14.747 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.747+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 state=0x55d79ccdb540 s->prefetch_data=0 2019-12-08T13:07:14.747 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.747+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 : miss 2019-12-08T13:07:14.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 info.flags=0x16 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 to cache LRU end 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:14.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:14.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.766+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:14.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.785+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=321 2019-12-08T13:07:14.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.785+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 info.flags=0x13 2019-12-08T13:07:14.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.785+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 to cache LRU end 2019-12-08T13:07:14.787 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.785+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:14.787 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.785+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:14.787 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.785+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 2019-12-08T13:07:14.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.797+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[9161132f-da48-40b4-a853-f341a1971d48.4507.2]) start [] num_entries 1100 2019-12-08T13:07:14.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.812+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-08T13:07:14.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.812+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55d79ccdbd08 s->prefetch_data=0 2019-12-08T13:07:14.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.819+0000 7fa10cf1bec0 10 manifest: total_size = 1848 2019-12-08T13:07:14.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.819+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.20 2019-12-08T13:07:14.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.819+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55d79ccdbd08 s->prefetch_data=0 2019-12-08T13:07:14.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.819+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd08 2019-12-08T13:07:14.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.819+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.2 2019-12-08T13:07:14.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-08T13:07:14.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-08T13:07:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-08T13:07:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-08T13:07:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-08T13:07:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-08T13:07:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-08T13:07:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-12-08T13:07:14.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-08T13:07:14.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-08T13:07:14.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-12-08T13:07:14.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-08T13:07:14.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:14.948+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.20 2019-12-08T13:07:15.027 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.028+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[9161132f-da48-40b4-a853-f341a1971d48.4507.2]) start _multipart_[] num_entries 1001 2019-12-08T13:07:15.027 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.028+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:15.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.045+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[9161132f-da48-40b4-a853-f341a1971d48.4507.2])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:15.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.130+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x55d79ccdcad0 s->prefetch_data=0 2019-12-08T13:07:15.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.130+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:15.142 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-12-08T13:07:15.142 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-08T13:07:15.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:15.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:15.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:15.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-12-08T13:07:15.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.143+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-12-08T13:07:15.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.155+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715778 cookie 94384536394048 notifier 4678 bl.length()=191 2019-12-08T13:07:15.762 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.763+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.2 from cache 2019-12-08T13:07:15.762 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.763+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=248 2019-12-08T13:07:15.776 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:15.778+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715779 cookie 94384536390704 notifier 4678 bl.length()=248 2019-12-08T13:07:16.559 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.560+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x55d79ccd6b20 s->prefetch_data=0 2019-12-08T13:07:16.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.560+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-12-08T13:07:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.580+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-12-08T13:07:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.580+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-08T13:07:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.580+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.580+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:16.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.580+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:16.580 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.580+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.596+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=236 2019-12-08T13:07:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.596+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-12-08T13:07:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.596+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-08T13:07:16.596 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.596+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 state=0x55d79ccd6760 s->prefetch_data=0 2019-12-08T13:07:16.596 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.596+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 : miss 2019-12-08T13:07:16.616 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 info.flags=0x16 2019-12-08T13:07:16.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 to cache LRU end 2019-12-08T13:07:16.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:16.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:16.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:16.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:16.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:16.618 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:16.618 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.617+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:16.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.634+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=328 2019-12-08T13:07:16.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.634+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 info.flags=0x13 2019-12-08T13:07:16.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.634+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 to cache LRU end 2019-12-08T13:07:16.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.634+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:16.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.634+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:16.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.634+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 2019-12-08T13:07:16.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.651+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[9161132f-da48-40b4-a853-f341a1971d48.4507.3]) start [] num_entries 1100 2019-12-08T13:07:16.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.665+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-08T13:07:16.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.665+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55d79ccdbd08 s->prefetch_data=0 2019-12-08T13:07:16.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.674+0000 7fa10cf1bec0 10 manifest: total_size = 886297 2019-12-08T13:07:16.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.674+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.40 2019-12-08T13:07:16.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.674+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55d79ccdbd08 s->prefetch_data=0 2019-12-08T13:07:16.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.674+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79ccdbd08 2019-12-08T13:07:16.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.674+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.3 2019-12-08T13:07:16.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:16.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-08T13:07:16.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:16.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:16.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2019-12-08T13:07:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.773+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2019-12-08T13:07:16.775 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.774+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.40 2019-12-08T13:07:16.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.877+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[9161132f-da48-40b4-a853-f341a1971d48.4507.3]) start _multipart_[] num_entries 1001 2019-12-08T13:07:16.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.877+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:16.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.887+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[9161132f-da48-40b4-a853-f341a1971d48.4507.3])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.969+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x55d79ccdc6c0 s->prefetch_data=0 2019-12-08T13:07:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.969+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:16.979 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-12-08T13:07:16.979 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-08T13:07:16.979 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:16.980 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:16.980 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:16.980 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-12-08T13:07:16.980 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.980+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-12-08T13:07:16.988 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:16.989+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715780 cookie 94384536396960 notifier 4678 bl.length()=198 2019-12-08T13:07:17.139 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.141+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:9161132f-da48-40b4-a853-f341a1971d48.4507.3 from cache 2019-12-08T13:07:17.140 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.141+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=255 2019-12-08T13:07:17.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.196+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715776 cookie 94384536434160 notifier 4678 bl.length()=255 2019-12-08T13:07:17.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.636+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x55d79ccdcad0 s->prefetch_data=0 2019-12-08T13:07:17.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.636+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-12-08T13:07:17.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.649+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-12-08T13:07:17.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.649+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-08T13:07:17.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.649+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:17.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.649+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:17.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.649+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:17.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.649+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:17.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.664+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=231 2019-12-08T13:07:17.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.664+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-12-08T13:07:17.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.664+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-08T13:07:17.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.664+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 state=0x55d79ccd6760 s->prefetch_data=0 2019-12-08T13:07:17.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.664+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 : miss 2019-12-08T13:07:17.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 info.flags=0x16 2019-12-08T13:07:17.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 to cache LRU end 2019-12-08T13:07:17.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:17.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:17.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:17.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:17.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:17.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:17.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.678+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:17.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.700+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=323 2019-12-08T13:07:17.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.700+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 info.flags=0x13 2019-12-08T13:07:17.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.700+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 to cache LRU end 2019-12-08T13:07:17.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.700+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:17.699 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.700+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:17.699 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.700+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:17.709 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.711+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[9161132f-da48-40b4-a853-f341a1971d48.4507.4]) start [] num_entries 1100 2019-12-08T13:07:17.731 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.733+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo[eqG4T9ENvKI19rJhOFHazSdfSsYfysa] 2019-12-08T13:07:17.731 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.733+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:17.742 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.744+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55d79cce0718 s->prefetch_data=0 2019-12-08T13:07:17.749 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.751+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 5nkb20cnrxhe0fza5egzfdwglgkuu3dz 2019-12-08T13:07:17.749 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.751+0000 7fa10cf1bec0 20 get_obj_state_impl: setting s->olh_tag to fy04qepzo3qs91xo045a15umufqdbvq7 2019-12-08T13:07:17.749 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.751+0000 7fa10cf1bec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=fy04qepzo3qs91xo045a15umufqdbvq7 2019-12-08T13:07:17.817 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.819+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:17.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.892+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:17.891 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.892+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:17.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.916+0000 7fa10cf1bec0 20 olh_log_entry: epoch=3 op=2 key=[] 2019-12-08T13:07:17.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.916+0000 7fa10cf1bec0 20 olh_log_entry: epoch=3 op=3 key=foo[eqG4T9ENvKI19rJhOFHazSdfSsYfysa] 2019-12-08T13:07:17.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.916+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-index-versioning.1:_:eqG4T9ENvKI19rJhOFHazSdfSsYfysa_foo state=0x55d79cce89b8 s->prefetch_data=0 2019-12-08T13:07:17.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.926+0000 7fa10cf1bec0 10 manifest: total_size = 886297 2019-12-08T13:07:17.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.926+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.55 2019-12-08T13:07:17.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.926+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-index-versioning.1:_:eqG4T9ENvKI19rJhOFHazSdfSsYfysa_foo state=0x55d79cce89b8 s->prefetch_data=0 2019-12-08T13:07:17.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.926+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cce89b8 2019-12-08T13:07:17.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:17.926+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:18.064 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:18.064 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-08T13:07:18.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:18.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:18.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:18.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2019-12-08T13:07:18.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:18.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2019-12-08T13:07:18.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2019-12-08T13:07:18.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2019-12-08T13:07:18.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2019-12-08T13:07:18.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2019-12-08T13:07:18.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.066+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.55 2019-12-08T13:07:18.193 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.195+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:18.193 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.195+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:18.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.334+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:18.391 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.393+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.4 2019-12-08T13:07:18.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.393+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[9161132f-da48-40b4-a853-f341a1971d48.4507.4]) start _multipart_[] num_entries 1001 2019-12-08T13:07:18.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.393+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:18.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.403+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[9161132f-da48-40b4-a853-f341a1971d48.4507.4])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:18.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.475+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x55d79ccdbcb0 s->prefetch_data=0 2019-12-08T13:07:18.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.475+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.487+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.487+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.487+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.487+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.487+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.488+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-12-08T13:07:18.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.488+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-12-08T13:07:18.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:18.505+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715781 cookie 94384536396960 notifier 4678 bl.length()=193 2019-12-08T13:07:19.026 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.028+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:9161132f-da48-40b4-a853-f341a1971d48.4507.4 from cache 2019-12-08T13:07:19.026 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.028+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=250 2019-12-08T13:07:19.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.043+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715779 cookie 94384536394048 notifier 4678 bl.length()=250 2019-12-08T13:07:19.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.292+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55d79ccdcad0 s->prefetch_data=0 2019-12-08T13:07:19.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.292+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-12-08T13:07:19.304 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.306+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-12-08T13:07:19.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.306+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-08T13:07:19.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.306+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:19.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.306+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:19.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.306+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:19.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.306+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:19.316 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.318+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-08T13:07:19.316 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.318+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-12-08T13:07:19.316 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.318+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-08T13:07:19.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.318+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 state=0x55d79ccdc3d0 s->prefetch_data=0 2019-12-08T13:07:19.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.318+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 : miss 2019-12-08T13:07:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 info.flags=0x16 2019-12-08T13:07:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 to cache LRU end 2019-12-08T13:07:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:19.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.338+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.358+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-08T13:07:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.358+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 info.flags=0x13 2019-12-08T13:07:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.358+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 to cache LRU end 2019-12-08T13:07:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.358+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:19.357 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.358+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:19.357 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.358+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 2019-12-08T13:07:19.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.372+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[9161132f-da48-40b4-a853-f341a1971d48.4507.5]) start [] num_entries 1100 2019-12-08T13:07:19.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.394+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-08T13:07:19.393 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.394+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55d79cce0718 s->prefetch_data=0 2019-12-08T13:07:19.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.408+0000 7fa10cf1bec0 10 manifest: total_size = 74198 2019-12-08T13:07:19.407 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.408+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.71 2019-12-08T13:07:19.407 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.408+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55d79cce0718 s->prefetch_data=0 2019-12-08T13:07:19.407 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.408+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cce0718 2019-12-08T13:07:19.407 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.408+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.5 2019-12-08T13:07:19.539 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-08T13:07:19.539 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-08T13:07:19.539 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-08T13:07:19.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-08T13:07:19.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-08T13:07:19.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-12-08T13:07:19.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-08T13:07:19.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-12-08T13:07:19.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-12-08T13:07:19.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-12-08T13:07:19.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-12-08T13:07:19.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-12-08T13:07:19.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.540+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.71 2019-12-08T13:07:19.658 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.660+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[9161132f-da48-40b4-a853-f341a1971d48.4507.5]) start _multipart_[] num_entries 1001 2019-12-08T13:07:19.658 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.660+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:19.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.714+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[9161132f-da48-40b4-a853-f341a1971d48.4507.5])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:19.817 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.819+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55d79ccdc3d0 s->prefetch_data=0 2019-12-08T13:07:19.817 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.819+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:19.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-12-08T13:07:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-08T13:07:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:19.837 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-12-08T13:07:19.837 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.835+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-12-08T13:07:19.852 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:19.854+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715779 cookie 94384536414592 notifier 4678 bl.length()=183 2019-12-08T13:07:20.440 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:20.442+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:9161132f-da48-40b4-a853-f341a1971d48.4507.5 from cache 2019-12-08T13:07:20.440 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:20.442+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=240 2019-12-08T13:07:20.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:20.465+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715777 cookie 94384536399872 notifier 4678 bl.length()=240 2019-12-08T13:07:21.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.174+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x55d79ccdcad0 s->prefetch_data=0 2019-12-08T13:07:21.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.174+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-12-08T13:07:21.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.189+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-12-08T13:07:21.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.189+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-08T13:07:21.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.189+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:21.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.189+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:21.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.189+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:21.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.189+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:21.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.198+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=227 2019-12-08T13:07:21.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.198+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-12-08T13:07:21.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.198+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-08T13:07:21.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.198+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 state=0x55d79ccdc390 s->prefetch_data=0 2019-12-08T13:07:21.196 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.198+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 : miss 2019-12-08T13:07:21.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 info.flags=0x16 2019-12-08T13:07:21.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 to cache LRU end 2019-12-08T13:07:21.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:21.215 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:21.215 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:21.215 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:21.215 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:21.215 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:21.215 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.216+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:21.232 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.234+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=319 2019-12-08T13:07:21.232 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.234+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 info.flags=0x13 2019-12-08T13:07:21.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.234+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 to cache LRU end 2019-12-08T13:07:21.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.234+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:21.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.234+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:21.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.234+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:21.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.257+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[9161132f-da48-40b4-a853-f341a1971d48.4507.7]) start [] num_entries 1100 2019-12-08T13:07:21.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-08T13:07:21.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-08T13:07:21.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-08T13:07:21.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-08T13:07:21.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-08T13:07:21.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-08T13:07:21.290 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.290+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55d79cc621e8 s->prefetch_data=0 2019-12-08T13:07:21.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.299+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:21.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.299+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.108 2019-12-08T13:07:21.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.299+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55d79cc621e8 s->prefetch_data=0 2019-12-08T13:07:21.298 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.299+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc621e8 2019-12-08T13:07:21.298 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.299+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:21.421 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.423+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:21.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.423+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:21.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.423+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:21.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.423+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:21.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.423+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.108 2019-12-08T13:07:21.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.664+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:21.680 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.682+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:21.680 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.682+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.109 2019-12-08T13:07:21.680 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.682+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:21.681 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.682+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:21.681 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.682+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:21.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.831+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:21.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.831+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:21.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.831+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:21.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.831+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:21.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.831+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.109 2019-12-08T13:07:21.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.945+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:21.956 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.958+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:21.956 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.958+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _xckKzPdzBjQrBA8nLr74exDmUsr-3-- 2019-12-08T13:07:21.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.958+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:21.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.958+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:21.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:21.958+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:22.098 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.100+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.098 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.100+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.098 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.100+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.100+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.100+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _xckKzPdzBjQrBA8nLr74exDmUsr-3-- 2019-12-08T13:07:22.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.187+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.196+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:22.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.196+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _i1Qy18Cc6lBeVIwvgv6tyFgk-KruHgx 2019-12-08T13:07:22.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.196+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.196+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:22.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.196+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:22.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.315+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.315+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.315+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _i1Qy18Cc6lBeVIwvgv6tyFgk-KruHgx 2019-12-08T13:07:22.377 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.379+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.391 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.393+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:22.391 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.393+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.112 2019-12-08T13:07:22.391 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.393+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.393+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:22.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.393+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:22.510 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.512+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.511 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.512+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.511 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.512+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.511 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.512+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.511 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.512+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.112 2019-12-08T13:07:22.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.577+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.585 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.587+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:22.585 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.587+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _yPSUyCpQC60FN3PBLyUQVx6v_u8J7WT 2019-12-08T13:07:22.585 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.587+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.587+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:22.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.587+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:22.675 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.677+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.677+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.677+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.677+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.677+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: _yPSUyCpQC60FN3PBLyUQVx6v_u8J7WT 2019-12-08T13:07:22.769 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.770+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.779 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.781+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:22.779 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.781+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.114 2019-12-08T13:07:22.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.781+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:22.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.781+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:22.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.781+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.889+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.889+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.889+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.889+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:22.889+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.114 2019-12-08T13:07:23.057 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.059+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.069+0000 7fa10cf1bec0 10 manifest: total_size = 5242880 2019-12-08T13:07:23.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.069+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _QUm2ng25LM-V7OCkKUsPz-jl8KOL4fH 2019-12-08T13:07:23.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.069+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.069+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:23.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.069+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:23.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.190+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:23.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.190+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-08T13:07:23.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.190+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: _QUm2ng25LM-V7OCkKUsPz-jl8KOL4fH 2019-12-08T13:07:23.270 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.272+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.279 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.281+0000 7fa10cf1bec0 10 manifest: total_size = 27201 2019-12-08T13:07:23.279 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.281+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.116 2019-12-08T13:07:23.279 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.281+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.279 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.281+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:23.279 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.281+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:23.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.402+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.412 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.414+0000 7fa10cf1bec0 10 manifest: total_size = 27201 2019-12-08T13:07:23.412 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.414+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _NUoLqkMZegUgjjhUZQ7a13zIigpQf_2 2019-12-08T13:07:23.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.414+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.414+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:23.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.414+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:23.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.561+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:23.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.561+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=27201 stripe_ofs=27201 part_ofs=0 rule->part_size=0 2019-12-08T13:07:23.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.562+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: _NUoLqkMZegUgjjhUZQ7a13zIigpQf_2 2019-12-08T13:07:23.644 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.646+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.655+0000 7fa10cf1bec0 10 manifest: total_size = 27201 2019-12-08T13:07:23.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.655+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _OmEdPPC3Kc2Wq1ggqZxAEeQRu2MSvtX 2019-12-08T13:07:23.654 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.655+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.654 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.655+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:23.654 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.655+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:23.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.767+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:23.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.767+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=27201 stripe_ofs=27201 part_ofs=0 rule->part_size=0 2019-12-08T13:07:23.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.767+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _OmEdPPC3Kc2Wq1ggqZxAEeQRu2MSvtX 2019-12-08T13:07:23.862 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.864+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.878+0000 7fa10cf1bec0 10 manifest: total_size = 27201 2019-12-08T13:07:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.878+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _cZxFhBlYF4ivr46pJw5DeBeTpyMFyaE 2019-12-08T13:07:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.878+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.878+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:23.878+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.7 2019-12-08T13:07:24.031 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.033+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[9161132f-da48-40b4-a853-f341a1971d48.4507.7]) start _multipart_[] num_entries 1001 2019-12-08T13:07:24.031 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.033+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:24.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.110+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[9161132f-da48-40b4-a853-f341a1971d48.4507.7])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:24.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.166+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x55d79ccdc390 s->prefetch_data=0 2019-12-08T13:07:24.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.166+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:24.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-12-08T13:07:24.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-08T13:07:24.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:24.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:24.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:24.181 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-12-08T13:07:24.181 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.182+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-12-08T13:07:24.192 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.194+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715782 cookie 94384536396960 notifier 4678 bl.length()=189 2019-12-08T13:07:24.671 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.673+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.7 from cache 2019-12-08T13:07:24.672 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.673+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=246 2019-12-08T13:07:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:24.685+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715780 cookie 94384536394048 notifier 4678 bl.length()=246 2019-12-08T13:07:25.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.046+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55d79ccdc390 s->prefetch_data=0 2019-12-08T13:07:25.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.046+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-12-08T13:07:25.055 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.057+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-12-08T13:07:25.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.057+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-08T13:07:25.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.057+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:25.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.057+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:25.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.057+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:25.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.057+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:25.064 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.066+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=237 2019-12-08T13:07:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.067+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-12-08T13:07:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.067+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-08T13:07:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.067+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 state=0x55d79ccdd140 s->prefetch_data=0 2019-12-08T13:07:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.067+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 : miss 2019-12-08T13:07:25.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 info.flags=0x16 2019-12-08T13:07:25.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 to cache LRU end 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:25.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.078+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:25.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.109+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=329 2019-12-08T13:07:25.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.109+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 info.flags=0x13 2019-12-08T13:07:25.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.109+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 to cache LRU end 2019-12-08T13:07:25.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.109+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:25.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.109+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:25.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.109+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:25.124 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.126+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[9161132f-da48-40b4-a853-f341a1971d48.4507.9]) start [] num_entries 1100 2019-12-08T13:07:25.163 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-08T13:07:25.163 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-08T13:07:25.163 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-08T13:07:25.163 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-08T13:07:25.163 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-08T13:07:25.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-08T13:07:25.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-08T13:07:25.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-08T13:07:25.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-08T13:07:25.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-08T13:07:25.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-08T13:07:25.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-08T13:07:25.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.165+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.188+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:25.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.188+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.224 2019-12-08T13:07:25.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.188+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.187 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.188+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:25.187 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.188+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:25.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:25.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:25.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:25.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-08T13:07:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:25.285 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-08T13:07:25.285 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.284+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.224 2019-12-08T13:07:25.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.342+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.351+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.351+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.225 2019-12-08T13:07:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.351+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.351+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:25.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.351+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:25.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:25.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:25.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:25.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-08T13:07:25.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:25.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-08T13:07:25.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.443+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.225 2019-12-08T13:07:25.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.520+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.531+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:25.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.531+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _MEfm5tDRhsEbmHObP2aF3toswE_TVua 2019-12-08T13:07:25.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.531+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.531+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:25.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.531+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-08T13:07:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.628+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _MEfm5tDRhsEbmHObP2aF3toswE_TVua 2019-12-08T13:07:25.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.688+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.697+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:25.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.697+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _XMG4bVH-1iv1BRb4wBjFAllbeCSrKc3 2019-12-08T13:07:25.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.697+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.697+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:25.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.697+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.811+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.811+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-08T13:07:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.811+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.811+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-08T13:07:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.811+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _XMG4bVH-1iv1BRb4wBjFAllbeCSrKc3 2019-12-08T13:07:25.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.936+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.944+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:25.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.944+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.233 2019-12-08T13:07:25.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.944+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:25.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.944+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:25.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:25.944+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:26.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-08T13:07:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-08T13:07:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.047+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.233 2019-12-08T13:07:26.105 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.107+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.118+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:26.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.118+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _Zab00lBSJdav0fSW6Oenhr1OMqGgHD1 2019-12-08T13:07:26.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.118+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.117 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.118+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:26.117 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.118+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:26.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:26.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:26.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:26.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-08T13:07:26.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:26.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-08T13:07:26.238 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.238+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: _Zab00lBSJdav0fSW6Oenhr1OMqGgHD1 2019-12-08T13:07:26.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.315+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.327+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:26.326 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.327+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.235 2019-12-08T13:07:26.326 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.327+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.326 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.327+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:26.326 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.327+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:26.437 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.437 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:26.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:26.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:26.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:26.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-08T13:07:26.440 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.438+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.235 2019-12-08T13:07:26.521 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.523+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.533 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.535+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:26.533 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.535+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _mHnbE6zkNxakmsr6Xa8LAF7efj6ysxo 2019-12-08T13:07:26.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.535+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.535+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:26.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.535+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:26.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.685+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:26.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.685+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-08T13:07:26.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.685+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:26.684 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.685+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-08T13:07:26.684 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.685+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _mHnbE6zkNxakmsr6Xa8LAF7efj6ysxo 2019-12-08T13:07:26.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.784+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.798+0000 7fa10cf1bec0 10 manifest: total_size = 541681 2019-12-08T13:07:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.798+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.242 2019-12-08T13:07:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.798+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.798+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:26.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.798+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:26.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:26.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-08T13:07:26.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=49187 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2019-12-08T13:07:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.910+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.242 2019-12-08T13:07:26.993 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:26.995+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:27.001 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.003+0000 7fa10cf1bec0 10 manifest: total_size = 541681 2019-12-08T13:07:27.001 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.003+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _txstLzFKKBUulRwTm8q4kuuomQy1Vj_ 2019-12-08T13:07:27.002 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.003+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:27.002 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.003+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:27.002 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.003+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.128+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.128+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=0 rule->part_size=0 2019-12-08T13:07:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.128+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: _txstLzFKKBUulRwTm8q4kuuomQy1Vj_ 2019-12-08T13:07:27.193 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.195+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:27.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.204+0000 7fa10cf1bec0 10 manifest: total_size = 541681 2019-12-08T13:07:27.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.204+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to _cLyXgqqAoeyGK6RIbQ4ksYdjQyzxikO 2019-12-08T13:07:27.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.204+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:27.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.204+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:27.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.204+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:27.298 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.300+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:27.298 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.300+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=0 rule->part_size=0 2019-12-08T13:07:27.298 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.300+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _cLyXgqqAoeyGK6RIbQ4ksYdjQyzxikO 2019-12-08T13:07:27.348 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.350+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:27.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.363+0000 7fa10cf1bec0 10 manifest: total_size = 541681 2019-12-08T13:07:27.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.363+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.245 2019-12-08T13:07:27.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.363+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:27.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.363+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:27.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.363+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.9 2019-12-08T13:07:27.459 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:27.459 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-08T13:07:27.459 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:27.459 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:27.459 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:27.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2019-12-08T13:07:27.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2019-12-08T13:07:27.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=49187 2019-12-08T13:07:27.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2019-12-08T13:07:27.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.461+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.245 2019-12-08T13:07:27.506 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.508+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[9161132f-da48-40b4-a853-f341a1971d48.4507.9]) start _multipart_[] num_entries 1001 2019-12-08T13:07:27.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.508+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:27.522 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.524+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[9161132f-da48-40b4-a853-f341a1971d48.4507.9])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:27.573 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.575+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55d79ccdd140 s->prefetch_data=0 2019-12-08T13:07:27.573 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.575+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-12-08T13:07:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-08T13:07:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-12-08T13:07:27.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.584+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-12-08T13:07:27.604 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.606+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715778 cookie 94384536399872 notifier 4678 bl.length()=199 2019-12-08T13:07:27.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.707+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:9161132f-da48-40b4-a853-f341a1971d48.4507.9 from cache 2019-12-08T13:07:27.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.707+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=256 2019-12-08T13:07:27.724 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:27.726+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715783 cookie 94384536396960 notifier 4678 bl.length()=256 2019-12-08T13:07:28.178 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.180+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55d79ccdd140 s->prefetch_data=0 2019-12-08T13:07:28.178 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.180+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-12-08T13:07:28.197 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.199+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-12-08T13:07:28.197 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.199+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-08T13:07:28.197 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.199+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:28.197 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.199+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:28.198 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.199+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:28.198 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.199+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:28.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.216+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=232 2019-12-08T13:07:28.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.216+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-12-08T13:07:28.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.216+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-08T13:07:28.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.216+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 state=0x55d79ccdc390 s->prefetch_data=0 2019-12-08T13:07:28.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.216+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 : miss 2019-12-08T13:07:28.225 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 info.flags=0x16 2019-12-08T13:07:28.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 to cache LRU end 2019-12-08T13:07:28.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:28.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:28.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:28.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:28.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:28.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:28.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.227+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:28.244 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.246+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=324 2019-12-08T13:07:28.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.246+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 info.flags=0x13 2019-12-08T13:07:28.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.246+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 to cache LRU end 2019-12-08T13:07:28.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.246+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:28.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.246+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:28.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.246+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 2019-12-08T13:07:28.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.263+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[9161132f-da48-40b4-a853-f341a1971d48.4507.8]) start [] num_entries 1100 2019-12-08T13:07:28.281 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.283+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-08T13:07:28.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.283+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-08T13:07:28.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.283+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-08T13:07:28.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.283+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:28.293 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.295+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:28.293 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.295+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.196 2019-12-08T13:07:28.293 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.295+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:28.294 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.295+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:28.294 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.295+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.8 2019-12-08T13:07:28.472 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:28.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:28.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:28.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:28.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-08T13:07:28.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:28.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-08T13:07:28.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.474+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.196 2019-12-08T13:07:28.566 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.568+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55d79cce1c28 s->prefetch_data=0 2019-12-08T13:07:28.574 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.576+0000 7fa10cf1bec0 10 manifest: total_size = 11534336 2019-12-08T13:07:28.574 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.576+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.203 2019-12-08T13:07:28.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.576+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55d79cce1c28 s->prefetch_data=0 2019-12-08T13:07:28.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.576+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cce1c28 2019-12-08T13:07:28.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.576+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.8 2019-12-08T13:07:28.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:28.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:28.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-08T13:07:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-08T13:07:28.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-08T13:07:28.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:28.706 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-08T13:07:28.706 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-08T13:07:28.706 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-08T13:07:28.706 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.704+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.203 2019-12-08T13:07:28.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.798+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:28.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.809+0000 7fa10cf1bec0 10 manifest: total_size = 541681 2019-12-08T13:07:28.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.809+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.210 2019-12-08T13:07:28.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.809+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:28.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.809+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:28.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.809+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.8 2019-12-08T13:07:28.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.948+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:28.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-08T13:07:28.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:28.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-08T13:07:28.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-08T13:07:28.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2019-12-08T13:07:28.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2019-12-08T13:07:28.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=49187 2019-12-08T13:07:28.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2019-12-08T13:07:28.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:28.949+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.210 2019-12-08T13:07:29.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.019+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[9161132f-da48-40b4-a853-f341a1971d48.4507.8]) start _multipart_[] num_entries 1001 2019-12-08T13:07:29.017 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.019+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:29.029 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.031+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[9161132f-da48-40b4-a853-f341a1971d48.4507.8])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:29.086 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.088+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55d79ccdc390 s->prefetch_data=0 2019-12-08T13:07:29.086 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.088+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:29.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-12-08T13:07:29.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-08T13:07:29.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:29.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:29.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:29.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-12-08T13:07:29.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.106+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-12-08T13:07:29.122 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.123+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715784 cookie 94384536416240 notifier 4678 bl.length()=194 2019-12-08T13:07:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.235+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:9161132f-da48-40b4-a853-f341a1971d48.4507.8 from cache 2019-12-08T13:07:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.235+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=251 2019-12-08T13:07:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.253+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715785 cookie 94384536396960 notifier 4678 bl.length()=251 2019-12-08T13:07:29.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.635+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x55d79ccdd140 s->prefetch_data=0 2019-12-08T13:07:29.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.635+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-12-08T13:07:29.644 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.646+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-12-08T13:07:29.644 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.646+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-08T13:07:29.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.646+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:29.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.646+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:29.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.646+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:29.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.646+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:29.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.657+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=222 2019-12-08T13:07:29.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.657+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-12-08T13:07:29.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.657+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-08T13:07:29.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.657+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 state=0x55d79cce8f60 s->prefetch_data=0 2019-12-08T13:07:29.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.657+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 : miss 2019-12-08T13:07:29.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 info.flags=0x16 2019-12-08T13:07:29.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 to cache LRU end 2019-12-08T13:07:29.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:29.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:29.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:29.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:29.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:29.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:29.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.668+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.688+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=314 2019-12-08T13:07:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.688+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 info.flags=0x13 2019-12-08T13:07:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.688+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 to cache LRU end 2019-12-08T13:07:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.688+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.688+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.688+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 2019-12-08T13:07:29.699 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.701+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[9161132f-da48-40b4-a853-f341a1971d48.4507.6]) start [] num_entries 1100 2019-12-08T13:07:29.713 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.715+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-08T13:07:29.714 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.715+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-08T13:07:29.714 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.715+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-08T13:07:29.714 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.715+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55d79cce1ac8 s->prefetch_data=0 2019-12-08T13:07:29.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.729+0000 7fa10cf1bec0 10 manifest: total_size = 10485760 2019-12-08T13:07:29.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.729+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.84 2019-12-08T13:07:29.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.729+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55d79cce1ac8 s->prefetch_data=0 2019-12-08T13:07:29.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.729+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cce1ac8 2019-12-08T13:07:29.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.730+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.6 2019-12-08T13:07:29.899 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:29.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:29.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:29.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-08T13:07:29.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:29.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-08T13:07:29.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.901+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.84 2019-12-08T13:07:29.984 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.986+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55d79cce1ac8 s->prefetch_data=0 2019-12-08T13:07:29.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.997+0000 7fa10cf1bec0 10 manifest: total_size = 10485760 2019-12-08T13:07:29.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.997+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.85 2019-12-08T13:07:29.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.997+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55d79cce1ac8 s->prefetch_data=0 2019-12-08T13:07:29.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.997+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cce1ac8 2019-12-08T13:07:29.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:29.997+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.6 2019-12-08T13:07:30.109 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:30.109 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-08T13:07:30.110 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:30.110 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-08T13:07:30.110 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-08T13:07:30.110 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-08T13:07:30.110 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.111+0000 7fa10cf1bec0 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: 9161132f-da48-40b4-a853-f341a1971d48.4530.85 2019-12-08T13:07:30.229 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.231+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55d79cce1ac8 s->prefetch_data=0 2019-12-08T13:07:30.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.265+0000 7fa10cf1bec0 10 manifest: total_size = 648332 2019-12-08T13:07:30.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.265+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.86 2019-12-08T13:07:30.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.265+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55d79cce1ac8 s->prefetch_data=0 2019-12-08T13:07:30.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.265+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cce1ac8 2019-12-08T13:07:30.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.265+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.6 2019-12-08T13:07:30.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.426+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[9161132f-da48-40b4-a853-f341a1971d48.4507.6]) start _multipart_[] num_entries 1001 2019-12-08T13:07:30.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.426+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:30.496 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.498+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[9161132f-da48-40b4-a853-f341a1971d48.4507.6])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:30.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.579+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x55d79cc63470 s->prefetch_data=0 2019-12-08T13:07:30.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.579+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:30.592 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-12-08T13:07:30.592 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-08T13:07:30.592 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:30.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:30.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:30.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-12-08T13:07:30.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.594+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-12-08T13:07:30.611 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:30.613+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715786 cookie 94384536396960 notifier 4678 bl.length()=184 2019-12-08T13:07:31.032 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.034+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:9161132f-da48-40b4-a853-f341a1971d48.4507.6 from cache 2019-12-08T13:07:31.033 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.034+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=241 2019-12-08T13:07:31.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.047+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715777 cookie 94384536434160 notifier 4678 bl.length()=241 2019-12-08T13:07:31.565 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.567+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x55d79cc63470 s->prefetch_data=0 2019-12-08T13:07:31.566 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.567+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-12-08T13:07:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.579+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-12-08T13:07:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.579+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-08T13:07:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.579+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:31.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.579+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:31.578 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.579+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:31.578 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.579+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.588+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-08T13:07:31.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.588+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-12-08T13:07:31.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.588+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-08T13:07:31.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.588+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 state=0x55d79ccdd140 s->prefetch_data=0 2019-12-08T13:07:31.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.588+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 : miss 2019-12-08T13:07:31.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 info.flags=0x16 2019-12-08T13:07:31.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 to cache LRU end 2019-12-08T13:07:31.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:31.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:31.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:31.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:31.599 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:31.599 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:31.599 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.600+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:31.611 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.613+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-08T13:07:31.611 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.613+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 info.flags=0x13 2019-12-08T13:07:31.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.613+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 to cache LRU end 2019-12-08T13:07:31.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.613+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:31.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.613+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:31.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.613+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 2019-12-08T13:07:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.625+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[9161132f-da48-40b4-a853-f341a1971d48.4507.10]) start [] num_entries 1100 2019-12-08T13:07:31.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.639+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-08T13:07:31.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.639+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-08T13:07:31.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.639+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-08T13:07:31.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.639+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-08T13:07:31.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.639+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:31.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.649+0000 7fa10cf1bec0 10 manifest: total_size = 1035 2019-12-08T13:07:31.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.649+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.305 2019-12-08T13:07:31.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.649+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:31.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.649+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:31.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.649+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.10 2019-12-08T13:07:31.756 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.758+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:31.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.774+0000 7fa10cf1bec0 10 manifest: total_size = 1035 2019-12-08T13:07:31.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.774+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.307 2019-12-08T13:07:31.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.774+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:31.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.774+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:31.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.774+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.10 2019-12-08T13:07:31.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.924+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:31.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.943+0000 7fa10cf1bec0 10 manifest: total_size = 49187 2019-12-08T13:07:31.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.943+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.306 2019-12-08T13:07:31.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.943+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:31.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.943+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:31.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:31.943+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.10 2019-12-08T13:07:32.092 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.094+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:32.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.110+0000 7fa10cf1bec0 10 manifest: total_size = 1048576 2019-12-08T13:07:32.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.110+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.304 2019-12-08T13:07:32.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.110+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:32.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.110+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:32.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.110+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.10 2019-12-08T13:07:32.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.316+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[9161132f-da48-40b4-a853-f341a1971d48.4507.10]) start _multipart_[] num_entries 1001 2019-12-08T13:07:32.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.316+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:32.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.381+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[9161132f-da48-40b4-a853-f341a1971d48.4507.10])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:32.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.386+0000 7fa0ca7e4700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:07:32.468 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.470+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x55d79cc627e0 s->prefetch_data=0 2019-12-08T13:07:32.468 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.470+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:32.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-12-08T13:07:32.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-08T13:07:32.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:32.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:32.481 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:32.481 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-12-08T13:07:32.481 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.482+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-08T13:07:32.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.494+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715781 cookie 94384536394048 notifier 4678 bl.length()=181 2019-12-08T13:07:32.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.772+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:9161132f-da48-40b4-a853-f341a1971d48.4507.10 from cache 2019-12-08T13:07:32.771 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.772+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=239 2019-12-08T13:07:32.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:32.783+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715782 cookie 94384536394048 notifier 4678 bl.length()=239 2019-12-08T13:07:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.539+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x55d79cc627e0 s->prefetch_data=0 2019-12-08T13:07:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.539+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-12-08T13:07:33.547 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.549+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-12-08T13:07:33.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.549+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-08T13:07:33.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.549+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:33.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.549+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:33.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.549+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:33.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.549+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:33.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.562+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-08T13:07:33.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.562+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-12-08T13:07:33.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.562+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-08T13:07:33.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.562+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 state=0x55d79ccdc700 s->prefetch_data=0 2019-12-08T13:07:33.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.562+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 : miss 2019-12-08T13:07:33.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 info.flags=0x16 2019-12-08T13:07:33.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 to cache LRU end 2019-12-08T13:07:33.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:33.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:33.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:33.572 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:33.572 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:33.572 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:33.572 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.573+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:33.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.585+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-08T13:07:33.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.585+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 info.flags=0x13 2019-12-08T13:07:33.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.585+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 to cache LRU end 2019-12-08T13:07:33.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.585+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:33.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.585+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:33.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.585+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 2019-12-08T13:07:33.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.611+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[9161132f-da48-40b4-a853-f341a1971d48.4507.11]) start [] num_entries 1100 2019-12-08T13:07:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.638+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-08T13:07:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.638+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-08T13:07:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.638+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-08T13:07:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.638+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-08T13:07:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.638+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:33.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.654+0000 7fa10cf1bec0 10 manifest: total_size = 527 2019-12-08T13:07:33.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.654+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.312 2019-12-08T13:07:33.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.654+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:33.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.654+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:33.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.654+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.11 2019-12-08T13:07:33.793 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.795+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:33.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.806+0000 7fa10cf1bec0 10 manifest: total_size = 527 2019-12-08T13:07:33.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.806+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.314 2019-12-08T13:07:33.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.806+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:33.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.806+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:33.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.807+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.11 2019-12-08T13:07:33.992 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:33.994+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:34.008 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.010+0000 7fa10cf1bec0 10 manifest: total_size = 524288 2019-12-08T13:07:34.008 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.010+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.313 2019-12-08T13:07:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.010+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.010+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:34.009 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.010+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.11 2019-12-08T13:07:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.243+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:34.266 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.268+0000 7fa10cf1bec0 10 manifest: total_size = 24595 2019-12-08T13:07:34.266 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.268+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.311 2019-12-08T13:07:34.266 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.268+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:34.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.268+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:34.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.268+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.11 2019-12-08T13:07:34.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.436+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[9161132f-da48-40b4-a853-f341a1971d48.4507.11]) start _multipart_[] num_entries 1001 2019-12-08T13:07:34.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.437+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:34.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.520+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[9161132f-da48-40b4-a853-f341a1971d48.4507.11])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:34.588 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.590+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x55d79ccddd20 s->prefetch_data=0 2019-12-08T13:07:34.588 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.590+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-12-08T13:07:34.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.607+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-08T13:07:34.620 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.622+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715783 cookie 94384536394048 notifier 4678 bl.length()=181 2019-12-08T13:07:34.740 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.742+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:9161132f-da48-40b4-a853-f341a1971d48.4507.11 from cache 2019-12-08T13:07:34.740 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.742+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=239 2019-12-08T13:07:34.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:34.756+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715784 cookie 94384536394048 notifier 4678 bl.length()=239 2019-12-08T13:07:35.153 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.155+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55d79ccdc700 s->prefetch_data=0 2019-12-08T13:07:35.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.155+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-12-08T13:07:35.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.167+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-12-08T13:07:35.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.167+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-08T13:07:35.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.167+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:35.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.167+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:35.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.167+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-12-08T13:07:35.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.167+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:35.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.190+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-08T13:07:35.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.190+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-12-08T13:07:35.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.190+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-08T13:07:35.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.190+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976aed00 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 state=0x55d79ccddd20 s->prefetch_data=0 2019-12-08T13:07:35.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.190+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 : miss 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 info.flags=0x16 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 to cache LRU end 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 20 Read xattr: user.rgw.acl 2019-12-08T13:07:35.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 20 Read xattr: user.rgw.idtag 2019-12-08T13:07:35.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 : type miss (requested=0x13, cached=0x16) 2019-12-08T13:07:35.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.205+0000 7fa10cf1bec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:35.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.254+0000 7fa10cf1bec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-08T13:07:35.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.254+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 info.flags=0x13 2019-12-08T13:07:35.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.254+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 to cache LRU end 2019-12-08T13:07:35.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.254+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:35.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.254+0000 7fa10cf1bec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-08T13:07:35.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.254+0000 7fa10cf1bec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 2019-12-08T13:07:35.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.270+0000 7fa10cf1bec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[9161132f-da48-40b4-a853-f341a1971d48.4507.12]) start [] num_entries 1100 2019-12-08T13:07:35.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.337+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-08T13:07:35.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.337+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-08T13:07:35.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.337+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-08T13:07:35.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.337+0000 7fa10cf1bec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-08T13:07:35.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.337+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.352+0000 7fa10cf1bec0 10 manifest: total_size = 0 2019-12-08T13:07:35.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.352+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.319 2019-12-08T13:07:35.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.352+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.352+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:35.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.352+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.12 2019-12-08T13:07:35.465 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.466+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.481+0000 7fa10cf1bec0 10 manifest: total_size = 0 2019-12-08T13:07:35.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.481+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.321 2019-12-08T13:07:35.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.481+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.481+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:35.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.481+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.12 2019-12-08T13:07:35.684 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.686+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.697+0000 7fa10cf1bec0 10 manifest: total_size = 0 2019-12-08T13:07:35.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.697+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.320 2019-12-08T13:07:35.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.697+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.697+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:35.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.697+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.12 2019-12-08T13:07:35.837 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.839+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.858 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.860+0000 7fa10cf1bec0 10 manifest: total_size = 0 2019-12-08T13:07:35.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.860+0000 7fa10cf1bec0 20 get_obj_state: setting s->obj_tag to 9161132f-da48-40b4-a853-f341a1971d48.4530.318 2019-12-08T13:07:35.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.860+0000 7fa10cf1bec0 20 get_obj_state: rctx=0x7ffd976aed20 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55d79cc62d88 s->prefetch_data=0 2019-12-08T13:07:35.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.860+0000 7fa10cf1bec0 20 prepare_atomic_modification: state is not atomic. state=0x55d79cc62d88 2019-12-08T13:07:35.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:35.860+0000 7fa10cf1bec0 20 bucket index object: default.rgw.buckets.index:.dir.9161132f-da48-40b4-a853-f341a1971d48.4507.12 2019-12-08T13:07:36.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.042+0000 7fa10cf1bec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[9161132f-da48-40b4-a853-f341a1971d48.4507.12]) start _multipart_[] num_entries 1001 2019-12-08T13:07:36.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.042+0000 7fa10cf1bec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-08T13:07:36.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.109+0000 7fa10cf1bec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[9161132f-da48-40b4-a853-f341a1971d48.4507.12])"; objs.size()=0; is_truncated=0 2019-12-08T13:07:36.193 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.195+0000 7fa10cf1bec0 20 get_system_obj_state: rctx=0x7ffd976ae6b8 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55d79ccde070 s->prefetch_data=0 2019-12-08T13:07:36.193 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.195+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-12-08T13:07:36.210 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-12-08T13:07:36.210 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-08T13:07:36.210 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T13:07:36.211 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T13:07:36.211 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-12-08T13:07:36.211 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-12-08T13:07:36.211 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.212+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-08T13:07:36.222 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.224+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715785 cookie 94384536394048 notifier 4678 bl.length()=181 2019-12-08T13:07:36.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.876+0000 7fa10cf1bec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:9161132f-da48-40b4-a853-f341a1971d48.4507.12 from cache 2019-12-08T13:07:36.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.876+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=239 2019-12-08T13:07:36.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:36.885+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715786 cookie 94384536394048 notifier 4678 bl.length()=239 2019-12-08T13:07:37.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:37.550+0000 7fa10cf1bec0 10 removing key index: RVZOKJJOYTFJVAUIOGTN 2019-12-08T13:07:37.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:37.551+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=170 2019-12-08T13:07:37.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:37.562+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715787 cookie 94384536394048 notifier 4678 bl.length()=170 2019-12-08T13:07:37.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:37.865+0000 7fa10cf1bec0 10 removing email index: ragweed.client.0+test@test.test 2019-12-08T13:07:37.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:37.865+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-12-08T13:07:37.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:37.875+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715787 cookie 94384536396960 notifier 4678 bl.length()=182 2019-12-08T13:07:38.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.166+0000 7fa10cf1bec0 10 removing user buckets index 2019-12-08T13:07:38.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.166+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-12-08T13:07:38.178 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.180+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715788 cookie 94384536416240 notifier 4678 bl.length()=173 2019-12-08T13:07:38.732 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.734+0000 7fa10cf1bec0 10 removing user index: ragweed.client.0 2019-12-08T13:07:38.732 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.734+0000 7fa10cf1bec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-12-08T13:07:38.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.734+0000 7fa10cf1bec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-12-08T13:07:38.742 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.744+0000 7fa0df7fe700 10 RGWWatcher::handle_notify() notify_id 562640715780 cookie 94384536414592 notifier 4678 bl.length()=165 2019-12-08T13:07:38.996 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.998+0000 7fa10cf1bec0 20 remove_watcher() i=0 2019-12-08T13:07:38.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:38.998+0000 7fa10cf1bec0 2 removed watcher, disabling cache 2019-12-08T13:07:39.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.049+0000 7fa10cf1bec0 20 remove_watcher() i=1 2019-12-08T13:07:39.096 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.097+0000 7fa10cf1bec0 20 remove_watcher() i=2 2019-12-08T13:07:39.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.185+0000 7fa10cf1bec0 20 remove_watcher() i=3 2019-12-08T13:07:39.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.224+0000 7fa10cf1bec0 20 remove_watcher() i=4 2019-12-08T13:07:39.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.256+0000 7fa10cf1bec0 20 remove_watcher() i=5 2019-12-08T13:07:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.301+0000 7fa10cf1bec0 20 remove_watcher() i=6 2019-12-08T13:07:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:39.348+0000 7fa10cf1bec0 20 remove_watcher() i=7 2019-12-08T13:07:39.354 INFO:tasks.ragweed:Removing ragweed... 2019-12-08T13:07:39.354 INFO:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/ragweed 2019-12-08T13:07:39.461 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-12-08T13:07:39.492 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-12-08T13:07:39.492 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-12-08T13:07:39.492 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-12-08T13:07:39.493 INFO:teuthology.orchestra.run.smithi129:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-12-08T13:07:39.549 INFO:teuthology.orchestra.run.smithi129.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-12-08T13:07:43.361 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-12-08T13:07:43.361 INFO:teuthology.orchestra.run.smithi129:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-12-08T13:07:45.596 INFO:teuthology.orchestra.run.smithi129.stdout: Gradle will be installed under /opt/gradle 2019-12-08T13:07:45.667 INFO:teuthology.orchestra.run.smithi129.stderr:--2019-12-08 13:07:45-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-12-08T13:07:45.683 INFO:teuthology.orchestra.run.smithi129.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.191.9, 104.18.190.9, 2606:4700::6812:bf09, ... 2019-12-08T13:07:45.691 INFO:teuthology.orchestra.run.smithi129.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.191.9|:443... connected. 2019-12-08T13:07:45.722 INFO:teuthology.orchestra.run.smithi129.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-12-08T13:07:45.722 INFO:teuthology.orchestra.run.smithi129.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-12-08T13:07:45.722 INFO:teuthology.orchestra.run.smithi129.stderr:--2019-12-08 13:07:45-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-12-08T13:07:45.734 INFO:teuthology.orchestra.run.smithi129.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.159.20, 104.17.160.20, 2606:4700::6811:9f14, ... 2019-12-08T13:07:45.741 INFO:teuthology.orchestra.run.smithi129.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.159.20|:443... connected. 2019-12-08T13:07:45.780 INFO:teuthology.orchestra.run.smithi129.stderr:HTTP request sent, awaiting response... 200 OK 2019-12-08T13:07:45.780 INFO:teuthology.orchestra.run.smithi129.stderr:Length: 75323957 (72M) [application/zip] 2019-12-08T13:07:45.780 INFO:teuthology.orchestra.run.smithi129.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-12-08T13:07:45.780 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T13:07:45.788 INFO:teuthology.orchestra.run.smithi129.stderr: 0K .......... .......... .......... .......... .......... 0% 6.64M 11s 2019-12-08T13:07:45.788 INFO:teuthology.orchestra.run.smithi129.stderr: 50K .......... .......... .......... .......... .......... 0% 171M 6s 2019-12-08T13:07:45.794 INFO:teuthology.orchestra.run.smithi129.stderr: 100K .......... .......... .......... .......... .......... 0% 6.82M 7s 2019-12-08T13:07:45.802 INFO:teuthology.orchestra.run.smithi129.stderr: 150K .......... .......... .......... .......... .......... 0% 155M 6s 2019-12-08T13:07:45.803 INFO:teuthology.orchestra.run.smithi129.stderr: 200K .......... .......... .......... .......... .......... 0% 121M 5s 2019-12-08T13:07:45.803 INFO:teuthology.orchestra.run.smithi129.stderr: 250K .......... .......... .......... .......... .......... 0% 130M 4s 2019-12-08T13:07:45.803 INFO:teuthology.orchestra.run.smithi129.stderr: 300K .......... .......... .......... .......... .......... 0% 166M 3s 2019-12-08T13:07:45.803 INFO:teuthology.orchestra.run.smithi129.stderr: 350K .......... .......... .......... .......... .......... 0% 7.90M 4s 2019-12-08T13:07:45.803 INFO:teuthology.orchestra.run.smithi129.stderr: 400K .......... .......... .......... .......... .......... 0% 121M 4s 2019-12-08T13:07:45.812 INFO:teuthology.orchestra.run.smithi129.stderr: 450K .......... .......... .......... .......... .......... 0% 180M 3s 2019-12-08T13:07:45.812 INFO:teuthology.orchestra.run.smithi129.stderr: 500K .......... .......... .......... .......... .......... 0% 130M 3s 2019-12-08T13:07:45.812 INFO:teuthology.orchestra.run.smithi129.stderr: 550K .......... .......... .......... .......... .......... 0% 103M 3s 2019-12-08T13:07:45.812 INFO:teuthology.orchestra.run.smithi129.stderr: 600K .......... .......... .......... .......... .......... 0% 118M 3s 2019-12-08T13:07:45.812 INFO:teuthology.orchestra.run.smithi129.stderr: 650K .......... .......... .......... .......... .......... 0% 137M 3s 2019-12-08T13:07:45.813 INFO:teuthology.orchestra.run.smithi129.stderr: 700K .......... .......... .......... .......... .......... 1% 160M 2s 2019-12-08T13:07:45.813 INFO:teuthology.orchestra.run.smithi129.stderr: 750K .......... .......... .......... .......... .......... 1% 106M 2s 2019-12-08T13:07:45.813 INFO:teuthology.orchestra.run.smithi129.stderr: 800K .......... .......... .......... .......... .......... 1% 129M 2s 2019-12-08T13:07:45.813 INFO:teuthology.orchestra.run.smithi129.stderr: 850K .......... .......... .......... .......... .......... 1% 11.6M 2s 2019-12-08T13:07:45.813 INFO:teuthology.orchestra.run.smithi129.stderr: 900K .......... .......... .......... .......... .......... 1% 97.8M 2s 2019-12-08T13:07:47.212 INFO:teuthology.orchestra.run.smithi129.stderr: 950K .......... .......... .......... .......... .......... 1% 127M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1000K .......... .......... .......... .......... .......... 1% 181M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1050K .......... .......... .......... .......... .......... 1% 119M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1100K .......... .......... .......... .......... .......... 1% 126M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1150K .......... .......... .......... .......... .......... 1% 113M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1200K .......... .......... .......... .......... .......... 1% 176M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1250K .......... .......... .......... .......... .......... 1% 119M 2s 2019-12-08T13:07:47.213 INFO:teuthology.orchestra.run.smithi129.stderr: 1300K .......... .......... .......... .......... .......... 1% 143M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1350K .......... .......... .......... .......... .......... 1% 130M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1400K .......... .......... .......... .......... .......... 1% 135M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1450K .......... .......... .......... .......... .......... 2% 137M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1500K .......... .......... .......... .......... .......... 2% 161M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1550K .......... .......... .......... .......... .......... 2% 129M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1600K .......... .......... .......... .......... .......... 2% 183M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1650K .......... .......... .......... .......... .......... 2% 128M 2s 2019-12-08T13:07:47.214 INFO:teuthology.orchestra.run.smithi129.stderr: 1700K .......... .......... .......... .......... .......... 2% 139M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 1750K .......... .......... .......... .......... .......... 2% 136M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 1800K .......... .......... .......... .......... .......... 2% 140M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 1850K .......... .......... .......... .......... .......... 2% 211M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 1900K .......... .......... .......... .......... .......... 2% 256M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 1950K .......... .......... .......... .......... .......... 2% 95.1M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 2000K .......... .......... .......... .......... .......... 2% 271M 1s 2019-12-08T13:07:47.215 INFO:teuthology.orchestra.run.smithi129.stderr: 2050K .......... .......... .......... .......... .......... 2% 191M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2100K .......... .......... .......... .......... .......... 2% 161M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2150K .......... .......... .......... .......... .......... 2% 180M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2200K .......... .......... .......... .......... .......... 3% 193M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2250K .......... .......... .......... .......... .......... 3% 157M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2300K .......... .......... .......... .......... .......... 3% 185M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2350K .......... .......... .......... .......... .......... 3% 174M 1s 2019-12-08T13:07:47.216 INFO:teuthology.orchestra.run.smithi129.stderr: 2400K .......... .......... .......... .......... .......... 3% 271M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2450K .......... .......... .......... .......... .......... 3% 203M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2500K .......... .......... .......... .......... .......... 3% 173M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2550K .......... .......... .......... .......... .......... 3% 190M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2600K .......... .......... .......... .......... .......... 3% 208M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2650K .......... .......... .......... .......... .......... 3% 187M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2700K .......... .......... .......... .......... .......... 3% 171M 1s 2019-12-08T13:07:47.217 INFO:teuthology.orchestra.run.smithi129.stderr: 2750K .......... .......... .......... .......... .......... 3% 149M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 2800K .......... .......... .......... .......... .......... 3% 167M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 2850K .......... .......... .......... .......... .......... 3% 186M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 2900K .......... .......... .......... .......... .......... 4% 217M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 2950K .......... .......... .......... .......... .......... 4% 183M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 3000K .......... .......... .......... .......... .......... 4% 215M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 3050K .......... .......... .......... .......... .......... 4% 198M 1s 2019-12-08T13:07:47.218 INFO:teuthology.orchestra.run.smithi129.stderr: 3100K .......... .......... .......... .......... .......... 4% 209M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3150K .......... .......... .......... .......... .......... 4% 173M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3200K .......... .......... .......... .......... .......... 4% 205M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3250K .......... .......... .......... .......... .......... 4% 204M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3300K .......... .......... .......... .......... .......... 4% 206M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3350K .......... .......... .......... .......... .......... 4% 182M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3400K .......... .......... .......... .......... .......... 4% 207M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3450K .......... .......... .......... .......... .......... 4% 197M 1s 2019-12-08T13:07:47.219 INFO:teuthology.orchestra.run.smithi129.stderr: 3500K .......... .......... .......... .......... .......... 4% 193M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3550K .......... .......... .......... .......... .......... 4% 170M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3600K .......... .......... .......... .......... .......... 4% 214M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3650K .......... .......... .......... .......... .......... 5% 187M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3700K .......... .......... .......... .......... .......... 5% 205M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3750K .......... .......... .......... .......... .......... 5% 267M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3800K .......... .......... .......... .......... .......... 5% 321M 1s 2019-12-08T13:07:47.220 INFO:teuthology.orchestra.run.smithi129.stderr: 3850K .......... .......... .......... .......... .......... 5% 336M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 3900K .......... .......... .......... .......... .......... 5% 305M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 3950K .......... .......... .......... .......... .......... 5% 251M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 4000K .......... .......... .......... .......... .......... 5% 334M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 4050K .......... .......... .......... .......... .......... 5% 338M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 4100K .......... .......... .......... .......... .......... 5% 344M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 4150K .......... .......... .......... .......... .......... 5% 208M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 4200K .......... .......... .......... .......... .......... 5% 292M 1s 2019-12-08T13:07:47.221 INFO:teuthology.orchestra.run.smithi129.stderr: 4250K .......... .......... .......... .......... .......... 5% 282M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4300K .......... .......... .......... .......... .......... 5% 237M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4350K .......... .......... .......... .......... .......... 5% 182M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4400K .......... .......... .......... .......... .......... 6% 204M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4450K .......... .......... .......... .......... .......... 6% 188M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4500K .......... .......... .......... .......... .......... 6% 211M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4550K .......... .......... .......... .......... .......... 6% 167M 1s 2019-12-08T13:07:47.222 INFO:teuthology.orchestra.run.smithi129.stderr: 4600K .......... .......... .......... .......... .......... 6% 202M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4650K .......... .......... .......... .......... .......... 6% 204M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4700K .......... .......... .......... .......... .......... 6% 224M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4750K .......... .......... .......... .......... .......... 6% 176M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4800K .......... .......... .......... .......... .......... 6% 212M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4850K .......... .......... .......... .......... .......... 6% 207M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4900K .......... .......... .......... .......... .......... 6% 211M 1s 2019-12-08T13:07:47.223 INFO:teuthology.orchestra.run.smithi129.stderr: 4950K .......... .......... .......... .......... .......... 6% 173M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5000K .......... .......... .......... .......... .......... 6% 206M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5050K .......... .......... .......... .......... .......... 6% 211M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5100K .......... .......... .......... .......... .......... 7% 195M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5150K .......... .......... .......... .......... .......... 7% 164M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5200K .......... .......... .......... .......... .......... 7% 194M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5250K .......... .......... .......... .......... .......... 7% 192M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5300K .......... .......... .......... .......... .......... 7% 242M 1s 2019-12-08T13:07:47.224 INFO:teuthology.orchestra.run.smithi129.stderr: 5350K .......... .......... .......... .......... .......... 7% 302M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5400K .......... .......... .......... .......... .......... 7% 315M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5450K .......... .......... .......... .......... .......... 7% 317M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5500K .......... .......... .......... .......... .......... 7% 322M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5550K .......... .......... .......... .......... .......... 7% 177M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5600K .......... .......... .......... .......... .......... 7% 241M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5650K .......... .......... .......... .......... .......... 7% 318M 1s 2019-12-08T13:07:47.225 INFO:teuthology.orchestra.run.smithi129.stderr: 5700K .......... .......... .......... .......... .......... 7% 347M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 5750K .......... .......... .......... .......... .......... 7% 319M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 5800K .......... .......... .......... .......... .......... 7% 335M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 5850K .......... .......... .......... .......... .......... 8% 346M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 5900K .......... .......... .......... .......... .......... 8% 344M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 5950K .......... .......... .......... .......... .......... 8% 289M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 6000K .......... .......... .......... .......... .......... 8% 350M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 6050K .......... .......... .......... .......... .......... 8% 240M 1s 2019-12-08T13:07:47.226 INFO:teuthology.orchestra.run.smithi129.stderr: 6100K .......... .......... .......... .......... .......... 8% 215M 1s 2019-12-08T13:07:47.227 INFO:teuthology.orchestra.run.smithi129.stderr: 6150K .......... .......... .......... .......... .......... 8% 191M 1s 2019-12-08T13:07:47.227 INFO:teuthology.orchestra.run.smithi129.stderr: 6200K .......... .......... .......... .......... .......... 8% 348M 1s 2019-12-08T13:07:47.227 INFO:teuthology.orchestra.run.smithi129.stderr: 6250K .......... .......... .......... .......... .......... 8% 357M 1s 2019-12-08T13:07:47.227 INFO:teuthology.orchestra.run.smithi129.stderr: 6300K .......... .......... .......... .......... .......... 8% 358M 1s 2019-12-08T13:07:47.227 INFO:teuthology.orchestra.run.smithi129.stderr: 6350K .......... .......... .......... .......... .......... 8% 314M 1s 2019-12-08T13:07:47.227 INFO:teuthology.orchestra.run.smithi129.stderr: 6400K .......... .......... .......... .......... .......... 8% 332M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6450K .......... .......... .......... .......... .......... 8% 354M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6500K .......... .......... .......... .......... .......... 8% 343M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6550K .......... .......... .......... .......... .......... 8% 294M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6600K .......... .......... .......... .......... .......... 9% 354M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6650K .......... .......... .......... .......... .......... 9% 319M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6700K .......... .......... .......... .......... .......... 9% 293M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6750K .......... .......... .......... .......... .......... 9% 261M 1s 2019-12-08T13:07:47.228 INFO:teuthology.orchestra.run.smithi129.stderr: 6800K .......... .......... .......... .......... .......... 9% 320M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 6850K .......... .......... .......... .......... .......... 9% 292M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 6900K .......... .......... .......... .......... .......... 9% 276M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 6950K .......... .......... .......... .......... .......... 9% 260M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 7000K .......... .......... .......... .......... .......... 9% 283M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 7050K .......... .......... .......... .......... .......... 9% 280M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 7100K .......... .......... .......... .......... .......... 9% 295M 1s 2019-12-08T13:07:47.229 INFO:teuthology.orchestra.run.smithi129.stderr: 7150K .......... .......... .......... .......... .......... 9% 320M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7200K .......... .......... .......... .......... .......... 9% 351M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7250K .......... .......... .......... .......... .......... 9% 365M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7300K .......... .......... .......... .......... .......... 9% 315M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7350K .......... .......... .......... .......... .......... 10% 241M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7400K .......... .......... .......... .......... .......... 10% 300M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7450K .......... .......... .......... .......... .......... 10% 309M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7500K .......... .......... .......... .......... .......... 10% 328M 1s 2019-12-08T13:07:47.230 INFO:teuthology.orchestra.run.smithi129.stderr: 7550K .......... .......... .......... .......... .......... 10% 302M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7600K .......... .......... .......... .......... .......... 10% 342M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7650K .......... .......... .......... .......... .......... 10% 315M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7700K .......... .......... .......... .......... .......... 10% 314M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7750K .......... .......... .......... .......... .......... 10% 214M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7800K .......... .......... .......... .......... .......... 10% 296M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7850K .......... .......... .......... .......... .......... 10% 309M 1s 2019-12-08T13:07:47.231 INFO:teuthology.orchestra.run.smithi129.stderr: 7900K .......... .......... .......... .......... .......... 10% 361M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 7950K .......... .......... .......... .......... .......... 10% 274M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 8000K .......... .......... .......... .......... .......... 10% 312M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 8050K .......... .......... .......... .......... .......... 11% 267M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 8100K .......... .......... .......... .......... .......... 11% 281M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 8150K .......... .......... .......... .......... .......... 11% 295M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 8200K .......... .......... .......... .......... .......... 11% 337M 1s 2019-12-08T13:07:47.232 INFO:teuthology.orchestra.run.smithi129.stderr: 8250K .......... .......... .......... .......... .......... 11% 325M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8300K .......... .......... .......... .......... .......... 11% 338M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8350K .......... .......... .......... .......... .......... 11% 303M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8400K .......... .......... .......... .......... .......... 11% 327M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8450K .......... .......... .......... .......... .......... 11% 278M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8500K .......... .......... .......... .......... .......... 11% 268M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8550K .......... .......... .......... .......... .......... 11% 263M 0s 2019-12-08T13:07:47.233 INFO:teuthology.orchestra.run.smithi129.stderr: 8600K .......... .......... .......... .......... .......... 11% 378M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8650K .......... .......... .......... .......... .......... 11% 305M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8700K .......... .......... .......... .......... .......... 11% 294M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8750K .......... .......... .......... .......... .......... 11% 259M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8800K .......... .......... .......... .......... .......... 12% 289M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8850K .......... .......... .......... .......... .......... 12% 278M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8900K .......... .......... .......... .......... .......... 12% 302M 0s 2019-12-08T13:07:47.234 INFO:teuthology.orchestra.run.smithi129.stderr: 8950K .......... .......... .......... .......... .......... 12% 230M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9000K .......... .......... .......... .......... .......... 12% 284M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9050K .......... .......... .......... .......... .......... 12% 359M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9100K .......... .......... .......... .......... .......... 12% 342M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9150K .......... .......... .......... .......... .......... 12% 266M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9200K .......... .......... .......... .......... .......... 12% 292M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9250K .......... .......... .......... .......... .......... 12% 288M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9300K .......... .......... .......... .......... .......... 12% 290M 0s 2019-12-08T13:07:47.235 INFO:teuthology.orchestra.run.smithi129.stderr: 9350K .......... .......... .......... .......... .......... 12% 280M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9400K .......... .......... .......... .......... .......... 12% 345M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9450K .......... .......... .......... .......... .......... 12% 301M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9500K .......... .......... .......... .......... .......... 12% 291M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9550K .......... .......... .......... .......... .......... 13% 296M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9600K .......... .......... .......... .......... .......... 13% 270M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9650K .......... .......... .......... .......... .......... 13% 287M 0s 2019-12-08T13:07:47.236 INFO:teuthology.orchestra.run.smithi129.stderr: 9700K .......... .......... .......... .......... .......... 13% 303M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 9750K .......... .......... .......... .......... .......... 13% 264M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 9800K .......... .......... .......... .......... .......... 13% 347M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 9850K .......... .......... .......... .......... .......... 13% 296M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 9900K .......... .......... .......... .......... .......... 13% 301M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 9950K .......... .......... .......... .......... .......... 13% 296M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 10000K .......... .......... .......... .......... .......... 13% 372M 0s 2019-12-08T13:07:47.237 INFO:teuthology.orchestra.run.smithi129.stderr: 10050K .......... .......... .......... .......... .......... 13% 341M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10100K .......... .......... .......... .......... .......... 13% 385M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10150K .......... .......... .......... .......... .......... 13% 317M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10200K .......... .......... .......... .......... .......... 13% 282M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10250K .......... .......... .......... .......... .......... 14% 333M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10300K .......... .......... .......... .......... .......... 14% 318M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10350K .......... .......... .......... .......... .......... 14% 282M 0s 2019-12-08T13:07:47.238 INFO:teuthology.orchestra.run.smithi129.stderr: 10400K .......... .......... .......... .......... .......... 14% 297M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10450K .......... .......... .......... .......... .......... 14% 320M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10500K .......... .......... .......... .......... .......... 14% 302M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10550K .......... .......... .......... .......... .......... 14% 258M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10600K .......... .......... .......... .......... .......... 14% 302M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10650K .......... .......... .......... .......... .......... 14% 290M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10700K .......... .......... .......... .......... .......... 14% 336M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10750K .......... .......... .......... .......... .......... 14% 256M 0s 2019-12-08T13:07:47.239 INFO:teuthology.orchestra.run.smithi129.stderr: 10800K .......... .......... .......... .......... .......... 14% 317M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 10850K .......... .......... .......... .......... .......... 14% 306M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 10900K .......... .......... .......... .......... .......... 14% 305M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 10950K .......... .......... .......... .......... .......... 14% 256M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 11000K .......... .......... .......... .......... .......... 15% 322M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 11050K .......... .......... .......... .......... .......... 15% 297M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 11100K .......... .......... .......... .......... .......... 15% 282M 0s 2019-12-08T13:07:47.240 INFO:teuthology.orchestra.run.smithi129.stderr: 11150K .......... .......... .......... .......... .......... 15% 250M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11200K .......... .......... .......... .......... .......... 15% 300M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11250K .......... .......... .......... .......... .......... 15% 293M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11300K .......... .......... .......... .......... .......... 15% 345M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11350K .......... .......... .......... .......... .......... 15% 295M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11400K .......... .......... .......... .......... .......... 15% 306M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11450K .......... .......... .......... .......... .......... 15% 285M 0s 2019-12-08T13:07:47.241 INFO:teuthology.orchestra.run.smithi129.stderr: 11500K .......... .......... .......... .......... .......... 15% 265M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11550K .......... .......... .......... .......... .......... 15% 300M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11600K .......... .......... .......... .......... .......... 15% 318M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11650K .......... .......... .......... .......... .......... 15% 362M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11700K .......... .......... .......... .......... .......... 15% 315M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11750K .......... .......... .......... .......... .......... 16% 289M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11800K .......... .......... .......... .......... .......... 16% 376M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11850K .......... .......... .......... .......... .......... 16% 347M 0s 2019-12-08T13:07:47.242 INFO:teuthology.orchestra.run.smithi129.stderr: 11900K .......... .......... .......... .......... .......... 16% 285M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 11950K .......... .......... .......... .......... .......... 16% 274M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 12000K .......... .......... .......... .......... .......... 16% 356M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 12050K .......... .......... .......... .......... .......... 16% 337M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 12100K .......... .......... .......... .......... .......... 16% 329M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 12150K .......... .......... .......... .......... .......... 16% 271M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 12200K .......... .......... .......... .......... .......... 16% 311M 0s 2019-12-08T13:07:47.243 INFO:teuthology.orchestra.run.smithi129.stderr: 12250K .......... .......... .......... .......... .......... 16% 347M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12300K .......... .......... .......... .......... .......... 16% 353M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12350K .......... .......... .......... .......... .......... 16% 310M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12400K .......... .......... .......... .......... .......... 16% 369M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12450K .......... .......... .......... .......... .......... 16% 361M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12500K .......... .......... .......... .......... .......... 17% 285M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12550K .......... .......... .......... .......... .......... 17% 182M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12600K .......... .......... .......... .......... .......... 17% 206M 0s 2019-12-08T13:07:47.244 INFO:teuthology.orchestra.run.smithi129.stderr: 12650K .......... .......... .......... .......... .......... 17% 235M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 12700K .......... .......... .......... .......... .......... 17% 212M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 12750K .......... .......... .......... .......... .......... 17% 181M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 12800K .......... .......... .......... .......... .......... 17% 191M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 12850K .......... .......... .......... .......... .......... 17% 197M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 12900K .......... .......... .......... .......... .......... 17% 210M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 12950K .......... .......... .......... .......... .......... 17% 165M 0s 2019-12-08T13:07:47.245 INFO:teuthology.orchestra.run.smithi129.stderr: 13000K .......... .......... .......... .......... .......... 17% 199M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13050K .......... .......... .......... .......... .......... 17% 220M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13100K .......... .......... .......... .......... .......... 17% 205M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13150K .......... .......... .......... .......... .......... 17% 162M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13200K .......... .......... .......... .......... .......... 18% 276M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13250K .......... .......... .......... .......... .......... 18% 312M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13300K .......... .......... .......... .......... .......... 18% 352M 0s 2019-12-08T13:07:47.246 INFO:teuthology.orchestra.run.smithi129.stderr: 13350K .......... .......... .......... .......... .......... 18% 308M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13400K .......... .......... .......... .......... .......... 18% 386M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13450K .......... .......... .......... .......... .......... 18% 180M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13500K .......... .......... .......... .......... .......... 18% 248M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13550K .......... .......... .......... .......... .......... 18% 282M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13600K .......... .......... .......... .......... .......... 18% 222M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13650K .......... .......... .......... .......... .......... 18% 208M 0s 2019-12-08T13:07:47.247 INFO:teuthology.orchestra.run.smithi129.stderr: 13700K .......... .......... .......... .......... .......... 18% 203M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 13750K .......... .......... .......... .......... .......... 18% 174M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 13800K .......... .......... .......... .......... .......... 18% 219M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 13850K .......... .......... .......... .......... .......... 18% 222M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 13900K .......... .......... .......... .......... .......... 18% 233M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 13950K .......... .......... .......... .......... .......... 19% 246M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 14000K .......... .......... .......... .......... .......... 19% 347M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 14050K .......... .......... .......... .......... .......... 19% 344M 0s 2019-12-08T13:07:47.248 INFO:teuthology.orchestra.run.smithi129.stderr: 14100K .......... .......... .......... .......... .......... 19% 359M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14150K .......... .......... .......... .......... .......... 19% 312M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14200K .......... .......... .......... .......... .......... 19% 353M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14250K .......... .......... .......... .......... .......... 19% 359M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14300K .......... .......... .......... .......... .......... 19% 384M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14350K .......... .......... .......... .......... .......... 19% 327M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14400K .......... .......... .......... .......... .......... 19% 377M 0s 2019-12-08T13:07:47.249 INFO:teuthology.orchestra.run.smithi129.stderr: 14450K .......... .......... .......... .......... .......... 19% 346M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14500K .......... .......... .......... .......... .......... 19% 357M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14550K .......... .......... .......... .......... .......... 19% 290M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14600K .......... .......... .......... .......... .......... 19% 308M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14650K .......... .......... .......... .......... .......... 19% 329M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14700K .......... .......... .......... .......... .......... 20% 350M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14750K .......... .......... .......... .......... .......... 20% 331M 0s 2019-12-08T13:07:47.250 INFO:teuthology.orchestra.run.smithi129.stderr: 14800K .......... .......... .......... .......... .......... 20% 353M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 14850K .......... .......... .......... .......... .......... 20% 378M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 14900K .......... .......... .......... .......... .......... 20% 340M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 14950K .......... .......... .......... .......... .......... 20% 317M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 15000K .......... .......... .......... .......... .......... 20% 378M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 15050K .......... .......... .......... .......... .......... 20% 322M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 15100K .......... .......... .......... .......... .......... 20% 372M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 15150K .......... .......... .......... .......... .......... 20% 333M 0s 2019-12-08T13:07:47.251 INFO:teuthology.orchestra.run.smithi129.stderr: 15200K .......... .......... .......... .......... .......... 20% 384M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15250K .......... .......... .......... .......... .......... 20% 387M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15300K .......... .......... .......... .......... .......... 20% 365M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15350K .......... .......... .......... .......... .......... 20% 300M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15400K .......... .......... .......... .......... .......... 21% 384M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15450K .......... .......... .......... .......... .......... 21% 352M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15500K .......... .......... .......... .......... .......... 21% 347M 0s 2019-12-08T13:07:47.252 INFO:teuthology.orchestra.run.smithi129.stderr: 15550K .......... .......... .......... .......... .......... 21% 319M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15600K .......... .......... .......... .......... .......... 21% 349M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15650K .......... .......... .......... .......... .......... 21% 334M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15700K .......... .......... .......... .......... .......... 21% 367M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15750K .......... .......... .......... .......... .......... 21% 309M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15800K .......... .......... .......... .......... .......... 21% 355M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15850K .......... .......... .......... .......... .......... 21% 355M 0s 2019-12-08T13:07:47.253 INFO:teuthology.orchestra.run.smithi129.stderr: 15900K .......... .......... .......... .......... .......... 21% 383M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 15950K .......... .......... .......... .......... .......... 21% 320M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 16000K .......... .......... .......... .......... .......... 21% 341M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 16050K .......... .......... .......... .......... .......... 21% 371M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 16100K .......... .......... .......... .......... .......... 21% 388M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 16150K .......... .......... .......... .......... .......... 22% 314M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 16200K .......... .......... .......... .......... .......... 22% 369M 0s 2019-12-08T13:07:47.254 INFO:teuthology.orchestra.run.smithi129.stderr: 16250K .......... .......... .......... .......... .......... 22% 386M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16300K .......... .......... .......... .......... .......... 22% 371M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16350K .......... .......... .......... .......... .......... 22% 317M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16400K .......... .......... .......... .......... .......... 22% 362M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16450K .......... .......... .......... .......... .......... 22% 372M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16500K .......... .......... .......... .......... .......... 22% 379M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16550K .......... .......... .......... .......... .......... 22% 319M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16600K .......... .......... .......... .......... .......... 22% 374M 0s 2019-12-08T13:07:47.255 INFO:teuthology.orchestra.run.smithi129.stderr: 16650K .......... .......... .......... .......... .......... 22% 382M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 16700K .......... .......... .......... .......... .......... 22% 372M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 16750K .......... .......... .......... .......... .......... 22% 334M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 16800K .......... .......... .......... .......... .......... 22% 386M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 16850K .......... .......... .......... .......... .......... 22% 391M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 16900K .......... .......... .......... .......... .......... 23% 384M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 16950K .......... .......... .......... .......... .......... 23% 322M 0s 2019-12-08T13:07:47.256 INFO:teuthology.orchestra.run.smithi129.stderr: 17000K .......... .......... .......... .......... .......... 23% 383M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17050K .......... .......... .......... .......... .......... 23% 381M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17100K .......... .......... .......... .......... .......... 23% 372M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17150K .......... .......... .......... .......... .......... 23% 307M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17200K .......... .......... .......... .......... .......... 23% 372M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17250K .......... .......... .......... .......... .......... 23% 390M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17300K .......... .......... .......... .......... .......... 23% 385M 0s 2019-12-08T13:07:47.257 INFO:teuthology.orchestra.run.smithi129.stderr: 17350K .......... .......... .......... .......... .......... 23% 319M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17400K .......... .......... .......... .......... .......... 23% 385M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17450K .......... .......... .......... .......... .......... 23% 359M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17500K .......... .......... .......... .......... .......... 23% 398M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17550K .......... .......... .......... .......... .......... 23% 325M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17600K .......... .......... .......... .......... .......... 23% 405M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17650K .......... .......... .......... .......... .......... 24% 411M 0s 2019-12-08T13:07:47.258 INFO:teuthology.orchestra.run.smithi129.stderr: 17700K .......... .......... .......... .......... .......... 24% 395M 0s 2019-12-08T13:07:47.259 INFO:teuthology.orchestra.run.smithi129.stderr: 17750K .......... .......... .......... .......... .......... 24% 346M 0s 2019-12-08T13:07:47.259 INFO:teuthology.orchestra.run.smithi129.stderr: 17800K .......... .......... .......... .......... .......... 24% 389M 0s 2019-12-08T13:07:47.259 INFO:teuthology.orchestra.run.smithi129.stderr: 17850K .......... .......... .......... .......... .......... 24% 399M 0s 2019-12-08T13:07:47.259 INFO:teuthology.orchestra.run.smithi129.stderr: 17900K .......... .......... .......... .......... .......... 24% 399M 0s 2019-12-08T13:07:47.259 INFO:teuthology.orchestra.run.smithi129.stderr: 17950K .......... .......... .......... .......... .......... 24% 309M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18000K .......... .......... .......... .......... .......... 24% 401M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18050K .......... .......... .......... .......... .......... 24% 397M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18100K .......... .......... .......... .......... .......... 24% 372M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18150K .......... .......... .......... .......... .......... 24% 357M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18200K .......... .......... .......... .......... .......... 24% 397M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18250K .......... .......... .......... .......... .......... 24% 389M 0s 2019-12-08T13:07:47.260 INFO:teuthology.orchestra.run.smithi129.stderr: 18300K .......... .......... .......... .......... .......... 24% 401M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18350K .......... .......... .......... .......... .......... 25% 349M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18400K .......... .......... .......... .......... .......... 25% 400M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18450K .......... .......... .......... .......... .......... 25% 408M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18500K .......... .......... .......... .......... .......... 25% 408M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18550K .......... .......... .......... .......... .......... 25% 363M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18600K .......... .......... .......... .......... .......... 25% 403M 0s 2019-12-08T13:07:47.261 INFO:teuthology.orchestra.run.smithi129.stderr: 18650K .......... .......... .......... .......... .......... 25% 383M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 18700K .......... .......... .......... .......... .......... 25% 396M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 18750K .......... .......... .......... .......... .......... 25% 339M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 18800K .......... .......... .......... .......... .......... 25% 396M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 18850K .......... .......... .......... .......... .......... 25% 410M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 18900K .......... .......... .......... .......... .......... 25% 404M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 18950K .......... .......... .......... .......... .......... 25% 338M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 19000K .......... .......... .......... .......... .......... 25% 385M 0s 2019-12-08T13:07:47.262 INFO:teuthology.orchestra.run.smithi129.stderr: 19050K .......... .......... .......... .......... .......... 25% 379M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19100K .......... .......... .......... .......... .......... 26% 384M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19150K .......... .......... .......... .......... .......... 26% 354M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19200K .......... .......... .......... .......... .......... 26% 396M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19250K .......... .......... .......... .......... .......... 26% 411M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19300K .......... .......... .......... .......... .......... 26% 395M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19350K .......... .......... .......... .......... .......... 26% 276M 0s 2019-12-08T13:07:47.263 INFO:teuthology.orchestra.run.smithi129.stderr: 19400K .......... .......... .......... .......... .......... 26% 322M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19450K .......... .......... .......... .......... .......... 26% 333M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19500K .......... .......... .......... .......... .......... 26% 382M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19550K .......... .......... .......... .......... .......... 26% 360M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19600K .......... .......... .......... .......... .......... 26% 415M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19650K .......... .......... .......... .......... .......... 26% 351M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19700K .......... .......... .......... .......... .......... 26% 394M 0s 2019-12-08T13:07:47.264 INFO:teuthology.orchestra.run.smithi129.stderr: 19750K .......... .......... .......... .......... .......... 26% 327M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 19800K .......... .......... .......... .......... .......... 26% 395M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 19850K .......... .......... .......... .......... .......... 27% 398M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 19900K .......... .......... .......... .......... .......... 27% 401M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 19950K .......... .......... .......... .......... .......... 27% 354M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 20000K .......... .......... .......... .......... .......... 27% 384M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 20050K .......... .......... .......... .......... .......... 27% 396M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 20100K .......... .......... .......... .......... .......... 27% 400M 0s 2019-12-08T13:07:47.265 INFO:teuthology.orchestra.run.smithi129.stderr: 20150K .......... .......... .......... .......... .......... 27% 298M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20200K .......... .......... .......... .......... .......... 27% 333M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20250K .......... .......... .......... .......... .......... 27% 411M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20300K .......... .......... .......... .......... .......... 27% 395M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20350K .......... .......... .......... .......... .......... 27% 323M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20400K .......... .......... .......... .......... .......... 27% 409M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20450K .......... .......... .......... .......... .......... 27% 390M 0s 2019-12-08T13:07:47.266 INFO:teuthology.orchestra.run.smithi129.stderr: 20500K .......... .......... .......... .......... .......... 27% 406M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20550K .......... .......... .......... .......... .......... 28% 353M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20600K .......... .......... .......... .......... .......... 28% 395M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20650K .......... .......... .......... .......... .......... 28% 376M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20700K .......... .......... .......... .......... .......... 28% 313M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20750K .......... .......... .......... .......... .......... 28% 273M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20800K .......... .......... .......... .......... .......... 28% 295M 0s 2019-12-08T13:07:47.267 INFO:teuthology.orchestra.run.smithi129.stderr: 20850K .......... .......... .......... .......... .......... 28% 304M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 20900K .......... .......... .......... .......... .......... 28% 331M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 20950K .......... .......... .......... .......... .......... 28% 347M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 21000K .......... .......... .......... .......... .......... 28% 396M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 21050K .......... .......... .......... .......... .......... 28% 391M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 21100K .......... .......... .......... .......... .......... 28% 315M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 21150K .......... .......... .......... .......... .......... 28% 275M 0s 2019-12-08T13:07:47.268 INFO:teuthology.orchestra.run.smithi129.stderr: 21200K .......... .......... .......... .......... .......... 28% 325M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21250K .......... .......... .......... .......... .......... 28% 329M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21300K .......... .......... .......... .......... .......... 29% 314M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21350K .......... .......... .......... .......... .......... 29% 301M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21400K .......... .......... .......... .......... .......... 29% 397M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21450K .......... .......... .......... .......... .......... 29% 366M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21500K .......... .......... .......... .......... .......... 29% 333M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21550K .......... .......... .......... .......... .......... 29% 290M 0s 2019-12-08T13:07:47.269 INFO:teuthology.orchestra.run.smithi129.stderr: 21600K .......... .......... .......... .......... .......... 29% 342M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21650K .......... .......... .......... .......... .......... 29% 319M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21700K .......... .......... .......... .......... .......... 29% 312M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21750K .......... .......... .......... .......... .......... 29% 294M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21800K .......... .......... .......... .......... .......... 29% 334M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21850K .......... .......... .......... .......... .......... 29% 348M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21900K .......... .......... .......... .......... .......... 29% 322M 0s 2019-12-08T13:07:47.270 INFO:teuthology.orchestra.run.smithi129.stderr: 21950K .......... .......... .......... .......... .......... 29% 312M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22000K .......... .......... .......... .......... .......... 29% 359M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22050K .......... .......... .......... .......... .......... 30% 331M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22100K .......... .......... .......... .......... .......... 30% 331M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22150K .......... .......... .......... .......... .......... 30% 270M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22200K .......... .......... .......... .......... .......... 30% 312M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22250K .......... .......... .......... .......... .......... 30% 327M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22300K .......... .......... .......... .......... .......... 30% 327M 0s 2019-12-08T13:07:47.271 INFO:teuthology.orchestra.run.smithi129.stderr: 22350K .......... .......... .......... .......... .......... 30% 275M 0s 2019-12-08T13:07:47.272 INFO:teuthology.orchestra.run.smithi129.stderr: 22400K .......... .......... .......... .......... .......... 30% 353M 0s 2019-12-08T13:07:47.272 INFO:teuthology.orchestra.run.smithi129.stderr: 22450K .......... .......... .......... .......... .......... 30% 326M 0s 2019-12-08T13:07:47.272 INFO:teuthology.orchestra.run.smithi129.stderr: 22500K .......... .......... .......... .......... .......... 30% 303M 0s 2019-12-08T13:07:47.272 INFO:teuthology.orchestra.run.smithi129.stderr: 22550K .......... .......... .......... .......... .......... 30% 276M 0s 2019-12-08T13:07:47.272 INFO:teuthology.orchestra.run.smithi129.stderr: 22600K .......... .......... .......... .......... .......... 30% 320M 0s 2019-12-08T13:07:47.272 INFO:teuthology.orchestra.run.smithi129.stderr: 22650K .......... .......... .......... .......... .......... 30% 302M 0s 2019-12-08T13:07:47.273 INFO:teuthology.orchestra.run.smithi129.stderr: 22700K .......... .......... .......... .......... .......... 30% 324M 0s 2019-12-08T13:07:47.273 INFO:teuthology.orchestra.run.smithi129.stderr: 22750K .......... .......... .......... .......... .......... 30% 283M 0s 2019-12-08T13:07:47.273 INFO:teuthology.orchestra.run.smithi129.stderr: 22800K .......... .......... .......... .......... .......... 31% 332M 0s 2019-12-08T13:07:47.273 INFO:teuthology.orchestra.run.smithi129.stderr: 22850K .......... .......... .......... .......... .......... 31% 305M 0s 2019-12-08T13:07:47.273 INFO:teuthology.orchestra.run.smithi129.stderr: 22900K .......... .......... .......... .......... .......... 31% 282M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 22950K .......... .......... .......... .......... .......... 31% 275M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23000K .......... .......... .......... .......... .......... 31% 318M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23050K .......... .......... .......... .......... .......... 31% 320M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23100K .......... .......... .......... .......... .......... 31% 335M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23150K .......... .......... .......... .......... .......... 31% 262M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23200K .......... .......... .......... .......... .......... 31% 341M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23250K .......... .......... .......... .......... .......... 31% 382M 0s 2019-12-08T13:07:47.274 INFO:teuthology.orchestra.run.smithi129.stderr: 23300K .......... .......... .......... .......... .......... 31% 333M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23350K .......... .......... .......... .......... .......... 31% 274M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23400K .......... .......... .......... .......... .......... 31% 328M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23450K .......... .......... .......... .......... .......... 31% 347M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23500K .......... .......... .......... .......... .......... 32% 415M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23550K .......... .......... .......... .......... .......... 32% 341M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23600K .......... .......... .......... .......... .......... 32% 396M 0s 2019-12-08T13:07:47.275 INFO:teuthology.orchestra.run.smithi129.stderr: 23650K .......... .......... .......... .......... .......... 32% 311M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 23700K .......... .......... .......... .......... .......... 32% 343M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 23750K .......... .......... .......... .......... .......... 32% 287M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 23800K .......... .......... .......... .......... .......... 32% 397M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 23850K .......... .......... .......... .......... .......... 32% 396M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 23900K .......... .......... .......... .......... .......... 32% 376M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 23950K .......... .......... .......... .......... .......... 32% 352M 0s 2019-12-08T13:07:47.276 INFO:teuthology.orchestra.run.smithi129.stderr: 24000K .......... .......... .......... .......... .......... 32% 366M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24050K .......... .......... .......... .......... .......... 32% 382M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24100K .......... .......... .......... .......... .......... 32% 421M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24150K .......... .......... .......... .......... .......... 32% 282M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24200K .......... .......... .......... .......... .......... 32% 310M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24250K .......... .......... .......... .......... .......... 33% 310M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24300K .......... .......... .......... .......... .......... 33% 356M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24350K .......... .......... .......... .......... .......... 33% 334M 0s 2019-12-08T13:07:47.277 INFO:teuthology.orchestra.run.smithi129.stderr: 24400K .......... .......... .......... .......... .......... 33% 399M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24450K .......... .......... .......... .......... .......... 33% 409M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24500K .......... .......... .......... .......... .......... 33% 402M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24550K .......... .......... .......... .......... .......... 33% 291M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24600K .......... .......... .......... .......... .......... 33% 296M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24650K .......... .......... .......... .......... .......... 33% 323M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24700K .......... .......... .......... .......... .......... 33% 339M 0s 2019-12-08T13:07:47.278 INFO:teuthology.orchestra.run.smithi129.stderr: 24750K .......... .......... .......... .......... .......... 33% 296M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 24800K .......... .......... .......... .......... .......... 33% 336M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 24850K .......... .......... .......... .......... .......... 33% 326M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 24900K .......... .......... .......... .......... .......... 33% 320M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 24950K .......... .......... .......... .......... .......... 33% 292M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 25000K .......... .......... .......... .......... .......... 34% 352M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 25050K .......... .......... .......... .......... .......... 34% 322M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 25100K .......... .......... .......... .......... .......... 34% 325M 0s 2019-12-08T13:07:47.279 INFO:teuthology.orchestra.run.smithi129.stderr: 25150K .......... .......... .......... .......... .......... 34% 258M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25200K .......... .......... .......... .......... .......... 34% 325M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25250K .......... .......... .......... .......... .......... 34% 317M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25300K .......... .......... .......... .......... .......... 34% 325M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25350K .......... .......... .......... .......... .......... 34% 295M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25400K .......... .......... .......... .......... .......... 34% 330M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25450K .......... .......... .......... .......... .......... 34% 340M 0s 2019-12-08T13:07:47.280 INFO:teuthology.orchestra.run.smithi129.stderr: 25500K .......... .......... .......... .......... .......... 34% 304M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25550K .......... .......... .......... .......... .......... 34% 286M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25600K .......... .......... .......... .......... .......... 34% 358M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25650K .......... .......... .......... .......... .......... 34% 315M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25700K .......... .......... .......... .......... .......... 35% 321M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25750K .......... .......... .......... .......... .......... 35% 280M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25800K .......... .......... .......... .......... .......... 35% 351M 0s 2019-12-08T13:07:47.281 INFO:teuthology.orchestra.run.smithi129.stderr: 25850K .......... .......... .......... .......... .......... 35% 318M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 25900K .......... .......... .......... .......... .......... 35% 325M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 25950K .......... .......... .......... .......... .......... 35% 288M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 26000K .......... .......... .......... .......... .......... 35% 326M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 26050K .......... .......... .......... .......... .......... 35% 330M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 26100K .......... .......... .......... .......... .......... 35% 313M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 26150K .......... .......... .......... .......... .......... 35% 273M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 26200K .......... .......... .......... .......... .......... 35% 344M 0s 2019-12-08T13:07:47.282 INFO:teuthology.orchestra.run.smithi129.stderr: 26250K .......... .......... .......... .......... .......... 35% 324M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26300K .......... .......... .......... .......... .......... 35% 326M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26350K .......... .......... .......... .......... .......... 35% 299M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26400K .......... .......... .......... .......... .......... 35% 315M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26450K .......... .......... .......... .......... .......... 36% 329M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26500K .......... .......... .......... .......... .......... 36% 322M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26550K .......... .......... .......... .......... .......... 36% 278M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26600K .......... .......... .......... .......... .......... 36% 326M 0s 2019-12-08T13:07:47.283 INFO:teuthology.orchestra.run.smithi129.stderr: 26650K .......... .......... .......... .......... .......... 36% 316M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 26700K .......... .......... .......... .......... .......... 36% 338M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 26750K .......... .......... .......... .......... .......... 36% 291M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 26800K .......... .......... .......... .......... .......... 36% 328M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 26850K .......... .......... .......... .......... .......... 36% 340M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 26900K .......... .......... .......... .......... .......... 36% 336M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 26950K .......... .......... .......... .......... .......... 36% 287M 0s 2019-12-08T13:07:47.284 INFO:teuthology.orchestra.run.smithi129.stderr: 27000K .......... .......... .......... .......... .......... 36% 339M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27050K .......... .......... .......... .......... .......... 36% 304M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27100K .......... .......... .......... .......... .......... 36% 324M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27150K .......... .......... .......... .......... .......... 36% 287M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27200K .......... .......... .......... .......... .......... 37% 338M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27250K .......... .......... .......... .......... .......... 37% 335M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27300K .......... .......... .......... .......... .......... 37% 318M 0s 2019-12-08T13:07:47.285 INFO:teuthology.orchestra.run.smithi129.stderr: 27350K .......... .......... .......... .......... .......... 37% 261M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27400K .......... .......... .......... .......... .......... 37% 343M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27450K .......... .......... .......... .......... .......... 37% 322M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27500K .......... .......... .......... .......... .......... 37% 325M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27550K .......... .......... .......... .......... .......... 37% 295M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27600K .......... .......... .......... .......... .......... 37% 324M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27650K .......... .......... .......... .......... .......... 37% 339M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27700K .......... .......... .......... .......... .......... 37% 320M 0s 2019-12-08T13:07:47.286 INFO:teuthology.orchestra.run.smithi129.stderr: 27750K .......... .......... .......... .......... .......... 37% 269M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 27800K .......... .......... .......... .......... .......... 37% 304M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 27850K .......... .......... .......... .......... .......... 37% 350M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 27900K .......... .......... .......... .......... .......... 37% 300M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 27950K .......... .......... .......... .......... .......... 38% 262M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 28000K .......... .......... .......... .......... .......... 38% 280M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 28050K .......... .......... .......... .......... .......... 38% 280M 0s 2019-12-08T13:07:47.287 INFO:teuthology.orchestra.run.smithi129.stderr: 28100K .......... .......... .......... .......... .......... 38% 263M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28150K .......... .......... .......... .......... .......... 38% 232M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28200K .......... .......... .......... .......... .......... 38% 299M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28250K .......... .......... .......... .......... .......... 38% 298M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28300K .......... .......... .......... .......... .......... 38% 281M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28350K .......... .......... .......... .......... .......... 38% 235M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28400K .......... .......... .......... .......... .......... 38% 265M 0s 2019-12-08T13:07:47.288 INFO:teuthology.orchestra.run.smithi129.stderr: 28450K .......... .......... .......... .......... .......... 38% 263M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28500K .......... .......... .......... .......... .......... 38% 257M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28550K .......... .......... .......... .......... .......... 38% 213M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28600K .......... .......... .......... .......... .......... 38% 239M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28650K .......... .......... .......... .......... .......... 39% 261M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28700K .......... .......... .......... .......... .......... 39% 262M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28750K .......... .......... .......... .......... .......... 39% 230M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28800K .......... .......... .......... .......... .......... 39% 254M 0s 2019-12-08T13:07:47.289 INFO:teuthology.orchestra.run.smithi129.stderr: 28850K .......... .......... .......... .......... .......... 39% 233M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 28900K .......... .......... .......... .......... .......... 39% 248M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 28950K .......... .......... .......... .......... .......... 39% 205M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 29000K .......... .......... .......... .......... .......... 39% 253M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 29050K .......... .......... .......... .......... .......... 39% 271M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 29100K .......... .......... .......... .......... .......... 39% 244M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 29150K .......... .......... .......... .......... .......... 39% 222M 0s 2019-12-08T13:07:47.290 INFO:teuthology.orchestra.run.smithi129.stderr: 29200K .......... .......... .......... .......... .......... 39% 243M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29250K .......... .......... .......... .......... .......... 39% 269M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29300K .......... .......... .......... .......... .......... 39% 228M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29350K .......... .......... .......... .......... .......... 39% 206M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29400K .......... .......... .......... .......... .......... 40% 248M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29450K .......... .......... .......... .......... .......... 40% 255M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29500K .......... .......... .......... .......... .......... 40% 239M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29550K .......... .......... .......... .......... .......... 40% 213M 0s 2019-12-08T13:07:47.291 INFO:teuthology.orchestra.run.smithi129.stderr: 29600K .......... .......... .......... .......... .......... 40% 273M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29650K .......... .......... .......... .......... .......... 40% 252M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29700K .......... .......... .......... .......... .......... 40% 251M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29750K .......... .......... .......... .......... .......... 40% 205M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29800K .......... .......... .......... .......... .......... 40% 255M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29850K .......... .......... .......... .......... .......... 40% 255M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29900K .......... .......... .......... .......... .......... 40% 261M 0s 2019-12-08T13:07:47.292 INFO:teuthology.orchestra.run.smithi129.stderr: 29950K .......... .......... .......... .......... .......... 40% 230M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30000K .......... .......... .......... .......... .......... 40% 251M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30050K .......... .......... .......... .......... .......... 40% 258M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30100K .......... .......... .......... .......... .......... 40% 230M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30150K .......... .......... .......... .......... .......... 41% 211M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30200K .......... .......... .......... .......... .......... 41% 266M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30250K .......... .......... .......... .......... .......... 41% 262M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30300K .......... .......... .......... .......... .......... 41% 267M 0s 2019-12-08T13:07:47.293 INFO:teuthology.orchestra.run.smithi129.stderr: 30350K .......... .......... .......... .......... .......... 41% 245M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30400K .......... .......... .......... .......... .......... 41% 286M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30450K .......... .......... .......... .......... .......... 41% 292M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30500K .......... .......... .......... .......... .......... 41% 289M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30550K .......... .......... .......... .......... .......... 41% 221M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30600K .......... .......... .......... .......... .......... 41% 250M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30650K .......... .......... .......... .......... .......... 41% 269M 0s 2019-12-08T13:07:47.294 INFO:teuthology.orchestra.run.smithi129.stderr: 30700K .......... .......... .......... .......... .......... 41% 293M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 30750K .......... .......... .......... .......... .......... 41% 255M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 30800K .......... .......... .......... .......... .......... 41% 274M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 30850K .......... .......... .......... .......... .......... 42% 291M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 30900K .......... .......... .......... .......... .......... 42% 285M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 30950K .......... .......... .......... .......... .......... 42% 236M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 31000K .......... .......... .......... .......... .......... 42% 294M 0s 2019-12-08T13:07:47.295 INFO:teuthology.orchestra.run.smithi129.stderr: 31050K .......... .......... .......... .......... .......... 42% 254M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31100K .......... .......... .......... .......... .......... 42% 294M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31150K .......... .......... .......... .......... .......... 42% 253M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31200K .......... .......... .......... .......... .......... 42% 267M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31250K .......... .......... .......... .......... .......... 42% 275M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31300K .......... .......... .......... .......... .......... 42% 283M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31350K .......... .......... .......... .......... .......... 42% 218M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31400K .......... .......... .......... .......... .......... 42% 293M 0s 2019-12-08T13:07:47.296 INFO:teuthology.orchestra.run.smithi129.stderr: 31450K .......... .......... .......... .......... .......... 42% 288M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31500K .......... .......... .......... .......... .......... 42% 292M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31550K .......... .......... .......... .......... .......... 42% 258M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31600K .......... .......... .......... .......... .......... 43% 291M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31650K .......... .......... .......... .......... .......... 43% 276M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31700K .......... .......... .......... .......... .......... 43% 287M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31750K .......... .......... .......... .......... .......... 43% 227M 0s 2019-12-08T13:07:47.297 INFO:teuthology.orchestra.run.smithi129.stderr: 31800K .......... .......... .......... .......... .......... 43% 287M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 31850K .......... .......... .......... .......... .......... 43% 284M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 31900K .......... .......... .......... .......... .......... 43% 267M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 31950K .......... .......... .......... .......... .......... 43% 231M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 32000K .......... .......... .......... .......... .......... 43% 281M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 32050K .......... .......... .......... .......... .......... 43% 279M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 32100K .......... .......... .......... .......... .......... 43% 263M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 32150K .......... .......... .......... .......... .......... 43% 244M 0s 2019-12-08T13:07:47.298 INFO:teuthology.orchestra.run.smithi129.stderr: 32200K .......... .......... .......... .......... .......... 43% 293M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32250K .......... .......... .......... .......... .......... 43% 272M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32300K .......... .......... .......... .......... .......... 43% 292M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32350K .......... .......... .......... .......... .......... 44% 260M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32400K .......... .......... .......... .......... .......... 44% 265M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32450K .......... .......... .......... .......... .......... 44% 286M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32500K .......... .......... .......... .......... .......... 44% 289M 0s 2019-12-08T13:07:47.299 INFO:teuthology.orchestra.run.smithi129.stderr: 32550K .......... .......... .......... .......... .......... 44% 231M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32600K .......... .......... .......... .......... .......... 44% 290M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32650K .......... .......... .......... .......... .......... 44% 271M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32700K .......... .......... .......... .......... .......... 44% 296M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32750K .......... .......... .......... .......... .......... 44% 257M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32800K .......... .......... .......... .......... .......... 44% 281M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32850K .......... .......... .......... .......... .......... 44% 267M 0s 2019-12-08T13:07:47.300 INFO:teuthology.orchestra.run.smithi129.stderr: 32900K .......... .......... .......... .......... .......... 44% 285M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 32950K .......... .......... .......... .......... .......... 44% 235M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 33000K .......... .......... .......... .......... .......... 44% 263M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 33050K .......... .......... .......... .......... .......... 44% 283M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 33100K .......... .......... .......... .......... .......... 45% 290M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 33150K .......... .......... .......... .......... .......... 45% 241M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 33200K .......... .......... .......... .......... .......... 45% 287M 0s 2019-12-08T13:07:47.301 INFO:teuthology.orchestra.run.smithi129.stderr: 33250K .......... .......... .......... .......... .......... 45% 266M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33300K .......... .......... .......... .......... .......... 45% 272M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33350K .......... .......... .......... .......... .......... 45% 261M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33400K .......... .......... .......... .......... .......... 45% 244M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33450K .......... .......... .......... .......... .......... 45% 240M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33500K .......... .......... .......... .......... .......... 45% 261M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33550K .......... .......... .......... .......... .......... 45% 218M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33600K .......... .......... .......... .......... .......... 45% 252M 0s 2019-12-08T13:07:47.302 INFO:teuthology.orchestra.run.smithi129.stderr: 33650K .......... .......... .......... .......... .......... 45% 258M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 33700K .......... .......... .......... .......... .......... 45% 258M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 33750K .......... .......... .......... .......... .......... 45% 215M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 33800K .......... .......... .......... .......... .......... 46% 273M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 33850K .......... .......... .......... .......... .......... 46% 272M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 33900K .......... .......... .......... .......... .......... 46% 245M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 33950K .......... .......... .......... .......... .......... 46% 224M 0s 2019-12-08T13:07:47.303 INFO:teuthology.orchestra.run.smithi129.stderr: 34000K .......... .......... .......... .......... .......... 46% 255M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34050K .......... .......... .......... .......... .......... 46% 247M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34100K .......... .......... .......... .......... .......... 46% 262M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34150K .......... .......... .......... .......... .......... 46% 212M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34200K .......... .......... .......... .......... .......... 46% 273M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34250K .......... .......... .......... .......... .......... 46% 247M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34300K .......... .......... .......... .......... .......... 46% 264M 0s 2019-12-08T13:07:47.304 INFO:teuthology.orchestra.run.smithi129.stderr: 34350K .......... .......... .......... .......... .......... 46% 200M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34400K .......... .......... .......... .......... .......... 46% 225M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34450K .......... .......... .......... .......... .......... 46% 266M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34500K .......... .......... .......... .......... .......... 46% 255M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34550K .......... .......... .......... .......... .......... 47% 222M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34600K .......... .......... .......... .......... .......... 47% 248M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34650K .......... .......... .......... .......... .......... 47% 257M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34700K .......... .......... .......... .......... .......... 47% 247M 0s 2019-12-08T13:07:47.305 INFO:teuthology.orchestra.run.smithi129.stderr: 34750K .......... .......... .......... .......... .......... 47% 214M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 34800K .......... .......... .......... .......... .......... 47% 262M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 34850K .......... .......... .......... .......... .......... 47% 262M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 34900K .......... .......... .......... .......... .......... 47% 248M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 34950K .......... .......... .......... .......... .......... 47% 212M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 35000K .......... .......... .......... .......... .......... 47% 261M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 35050K .......... .......... .......... .......... .......... 47% 255M 0s 2019-12-08T13:07:47.306 INFO:teuthology.orchestra.run.smithi129.stderr: 35100K .......... .......... .......... .......... .......... 47% 239M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35150K .......... .......... .......... .......... .......... 47% 187M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35200K .......... .......... .......... .......... .......... 47% 251M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35250K .......... .......... .......... .......... .......... 47% 257M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35300K .......... .......... .......... .......... .......... 48% 261M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35350K .......... .......... .......... .......... .......... 48% 207M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35400K .......... .......... .......... .......... .......... 48% 278M 0s 2019-12-08T13:07:47.307 INFO:teuthology.orchestra.run.smithi129.stderr: 35450K .......... .......... .......... .......... .......... 48% 260M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35500K .......... .......... .......... .......... .......... 48% 241M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35550K .......... .......... .......... .......... .......... 48% 221M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35600K .......... .......... .......... .......... .......... 48% 231M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35650K .......... .......... .......... .......... .......... 48% 272M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35700K .......... .......... .......... .......... .......... 48% 266M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35750K .......... .......... .......... .......... .......... 48% 227M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35800K .......... .......... .......... .......... .......... 48% 254M 0s 2019-12-08T13:07:47.308 INFO:teuthology.orchestra.run.smithi129.stderr: 35850K .......... .......... .......... .......... .......... 48% 265M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 35900K .......... .......... .......... .......... .......... 48% 290M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 35950K .......... .......... .......... .......... .......... 48% 209M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 36000K .......... .......... .......... .......... .......... 49% 255M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 36050K .......... .......... .......... .......... .......... 49% 246M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 36100K .......... .......... .......... .......... .......... 49% 258M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 36150K .......... .......... .......... .......... .......... 49% 226M 0s 2019-12-08T13:07:47.309 INFO:teuthology.orchestra.run.smithi129.stderr: 36200K .......... .......... .......... .......... .......... 49% 260M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36250K .......... .......... .......... .......... .......... 49% 256M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36300K .......... .......... .......... .......... .......... 49% 256M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36350K .......... .......... .......... .......... .......... 49% 208M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36400K .......... .......... .......... .......... .......... 49% 261M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36450K .......... .......... .......... .......... .......... 49% 255M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36500K .......... .......... .......... .......... .......... 49% 247M 0s 2019-12-08T13:07:47.310 INFO:teuthology.orchestra.run.smithi129.stderr: 36550K .......... .......... .......... .......... .......... 49% 221M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36600K .......... .......... .......... .......... .......... 49% 254M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36650K .......... .......... .......... .......... .......... 49% 248M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36700K .......... .......... .......... .......... .......... 49% 243M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36750K .......... .......... .......... .......... .......... 50% 207M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36800K .......... .......... .......... .......... .......... 50% 269M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36850K .......... .......... .......... .......... .......... 50% 266M 0s 2019-12-08T13:07:47.311 INFO:teuthology.orchestra.run.smithi129.stderr: 36900K .......... .......... .......... .......... .......... 50% 252M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 36950K .......... .......... .......... .......... .......... 50% 221M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 37000K .......... .......... .......... .......... .......... 50% 266M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 37050K .......... .......... .......... .......... .......... 50% 248M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 37100K .......... .......... .......... .......... .......... 50% 262M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 37150K .......... .......... .......... .......... .......... 50% 239M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 37200K .......... .......... .......... .......... .......... 50% 258M 0s 2019-12-08T13:07:47.312 INFO:teuthology.orchestra.run.smithi129.stderr: 37250K .......... .......... .......... .......... .......... 50% 262M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37300K .......... .......... .......... .......... .......... 50% 268M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37350K .......... .......... .......... .......... .......... 50% 238M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37400K .......... .......... .......... .......... .......... 50% 274M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37450K .......... .......... .......... .......... .......... 50% 297M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37500K .......... .......... .......... .......... .......... 51% 286M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37550K .......... .......... .......... .......... .......... 51% 256M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37600K .......... .......... .......... .......... .......... 51% 284M 0s 2019-12-08T13:07:47.313 INFO:teuthology.orchestra.run.smithi129.stderr: 37650K .......... .......... .......... .......... .......... 51% 256M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 37700K .......... .......... .......... .......... .......... 51% 280M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 37750K .......... .......... .......... .......... .......... 51% 230M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 37800K .......... .......... .......... .......... .......... 51% 272M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 37850K .......... .......... .......... .......... .......... 51% 293M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 37900K .......... .......... .......... .......... .......... 51% 278M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 37950K .......... .......... .......... .......... .......... 51% 256M 0s 2019-12-08T13:07:47.314 INFO:teuthology.orchestra.run.smithi129.stderr: 38000K .......... .......... .......... .......... .......... 51% 281M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38050K .......... .......... .......... .......... .......... 51% 298M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38100K .......... .......... .......... .......... .......... 51% 288M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38150K .......... .......... .......... .......... .......... 51% 241M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38200K .......... .......... .......... .......... .......... 51% 284M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38250K .......... .......... .......... .......... .......... 52% 271M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38300K .......... .......... .......... .......... .......... 52% 285M 0s 2019-12-08T13:07:47.315 INFO:teuthology.orchestra.run.smithi129.stderr: 38350K .......... .......... .......... .......... .......... 52% 237M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38400K .......... .......... .......... .......... .......... 52% 268M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38450K .......... .......... .......... .......... .......... 52% 279M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38500K .......... .......... .......... .......... .......... 52% 284M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38550K .......... .......... .......... .......... .......... 52% 224M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38600K .......... .......... .......... .......... .......... 52% 244M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38650K .......... .......... .......... .......... .......... 52% 286M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38700K .......... .......... .......... .......... .......... 52% 288M 0s 2019-12-08T13:07:47.316 INFO:teuthology.orchestra.run.smithi129.stderr: 38750K .......... .......... .......... .......... .......... 52% 219M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 38800K .......... .......... .......... .......... .......... 52% 280M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 38850K .......... .......... .......... .......... .......... 52% 302M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 38900K .......... .......... .......... .......... .......... 52% 284M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 38950K .......... .......... .......... .......... .......... 53% 263M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 39000K .......... .......... .......... .......... .......... 53% 298M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 39050K .......... .......... .......... .......... .......... 53% 287M 0s 2019-12-08T13:07:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: 39100K .......... .......... .......... .......... .......... 53% 299M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39150K .......... .......... .......... .......... .......... 53% 237M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39200K .......... .......... .......... .......... .......... 53% 288M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39250K .......... .......... .......... .......... .......... 53% 287M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39300K .......... .......... .......... .......... .......... 53% 301M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39350K .......... .......... .......... .......... .......... 53% 214M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39400K .......... .......... .......... .......... .......... 53% 290M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39450K .......... .......... .......... .......... .......... 53% 287M 0s 2019-12-08T13:07:47.318 INFO:teuthology.orchestra.run.smithi129.stderr: 39500K .......... .......... .......... .......... .......... 53% 298M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39550K .......... .......... .......... .......... .......... 53% 233M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39600K .......... .......... .......... .......... .......... 53% 284M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39650K .......... .......... .......... .......... .......... 53% 273M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39700K .......... .......... .......... .......... .......... 54% 284M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39750K .......... .......... .......... .......... .......... 54% 261M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39800K .......... .......... .......... .......... .......... 54% 285M 0s 2019-12-08T13:07:47.319 INFO:teuthology.orchestra.run.smithi129.stderr: 39850K .......... .......... .......... .......... .......... 54% 292M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 39900K .......... .......... .......... .......... .......... 54% 248M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 39950K .......... .......... .......... .......... .......... 54% 225M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 40000K .......... .......... .......... .......... .......... 54% 271M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 40050K .......... .......... .......... .......... .......... 54% 293M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 40100K .......... .......... .......... .......... .......... 54% 243M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 40150K .......... .......... .......... .......... .......... 54% 207M 0s 2019-12-08T13:07:47.320 INFO:teuthology.orchestra.run.smithi129.stderr: 40200K .......... .......... .......... .......... .......... 54% 253M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40250K .......... .......... .......... .......... .......... 54% 251M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40300K .......... .......... .......... .......... .......... 54% 260M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40350K .......... .......... .......... .......... .......... 54% 215M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40400K .......... .......... .......... .......... .......... 54% 271M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40450K .......... .......... .......... .......... .......... 55% 262M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40500K .......... .......... .......... .......... .......... 55% 243M 0s 2019-12-08T13:07:47.321 INFO:teuthology.orchestra.run.smithi129.stderr: 40550K .......... .......... .......... .......... .......... 55% 238M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40600K .......... .......... .......... .......... .......... 55% 263M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40650K .......... .......... .......... .......... .......... 55% 247M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40700K .......... .......... .......... .......... .......... 55% 245M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40750K .......... .......... .......... .......... .......... 55% 219M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40800K .......... .......... .......... .......... .......... 55% 249M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40850K .......... .......... .......... .......... .......... 55% 243M 0s 2019-12-08T13:07:47.322 INFO:teuthology.orchestra.run.smithi129.stderr: 40900K .......... .......... .......... .......... .......... 55% 252M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 40950K .......... .......... .......... .......... .......... 55% 216M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41000K .......... .......... .......... .......... .......... 55% 268M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41050K .......... .......... .......... .......... .......... 55% 233M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41100K .......... .......... .......... .......... .......... 55% 287M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41150K .......... .......... .......... .......... .......... 56% 225M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41200K .......... .......... .......... .......... .......... 56% 243M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41250K .......... .......... .......... .......... .......... 56% 245M 0s 2019-12-08T13:07:47.323 INFO:teuthology.orchestra.run.smithi129.stderr: 41300K .......... .......... .......... .......... .......... 56% 263M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41350K .......... .......... .......... .......... .......... 56% 217M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41400K .......... .......... .......... .......... .......... 56% 249M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41450K .......... .......... .......... .......... .......... 56% 249M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41500K .......... .......... .......... .......... .......... 56% 264M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41550K .......... .......... .......... .......... .......... 56% 212M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41600K .......... .......... .......... .......... .......... 56% 266M 0s 2019-12-08T13:07:47.324 INFO:teuthology.orchestra.run.smithi129.stderr: 41650K .......... .......... .......... .......... .......... 56% 233M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 41700K .......... .......... .......... .......... .......... 56% 234M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 41750K .......... .......... .......... .......... .......... 56% 216M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 41800K .......... .......... .......... .......... .......... 56% 257M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 41850K .......... .......... .......... .......... .......... 56% 255M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 41900K .......... .......... .......... .......... .......... 57% 225M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 41950K .......... .......... .......... .......... .......... 57% 213M 0s 2019-12-08T13:07:47.325 INFO:teuthology.orchestra.run.smithi129.stderr: 42000K .......... .......... .......... .......... .......... 57% 271M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42050K .......... .......... .......... .......... .......... 57% 239M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42100K .......... .......... .......... .......... .......... 57% 260M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42150K .......... .......... .......... .......... .......... 57% 220M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42200K .......... .......... .......... .......... .......... 57% 259M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42250K .......... .......... .......... .......... .......... 57% 267M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42300K .......... .......... .......... .......... .......... 57% 264M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42350K .......... .......... .......... .......... .......... 57% 207M 0s 2019-12-08T13:07:47.326 INFO:teuthology.orchestra.run.smithi129.stderr: 42400K .......... .......... .......... .......... .......... 57% 260M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42450K .......... .......... .......... .......... .......... 57% 278M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42500K .......... .......... .......... .......... .......... 57% 244M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42550K .......... .......... .......... .......... .......... 57% 228M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42600K .......... .......... .......... .......... .......... 57% 270M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42650K .......... .......... .......... .......... .......... 58% 251M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42700K .......... .......... .......... .......... .......... 58% 253M 0s 2019-12-08T13:07:47.327 INFO:teuthology.orchestra.run.smithi129.stderr: 42750K .......... .......... .......... .......... .......... 58% 208M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 42800K .......... .......... .......... .......... .......... 58% 253M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 42850K .......... .......... .......... .......... .......... 58% 262M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 42900K .......... .......... .......... .......... .......... 58% 255M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 42950K .......... .......... .......... .......... .......... 58% 219M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 43000K .......... .......... .......... .......... .......... 58% 267M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 43050K .......... .......... .......... .......... .......... 58% 260M 0s 2019-12-08T13:07:47.328 INFO:teuthology.orchestra.run.smithi129.stderr: 43100K .......... .......... .......... .......... .......... 58% 246M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43150K .......... .......... .......... .......... .......... 58% 203M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43200K .......... .......... .......... .......... .......... 58% 243M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43250K .......... .......... .......... .......... .......... 58% 255M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43300K .......... .......... .......... .......... .......... 58% 239M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43350K .......... .......... .......... .......... .......... 59% 231M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43400K .......... .......... .......... .......... .......... 59% 252M 0s 2019-12-08T13:07:47.329 INFO:teuthology.orchestra.run.smithi129.stderr: 43450K .......... .......... .......... .......... .......... 59% 250M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43500K .......... .......... .......... .......... .......... 59% 252M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43550K .......... .......... .......... .......... .......... 59% 203M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43600K .......... .......... .......... .......... .......... 59% 244M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43650K .......... .......... .......... .......... .......... 59% 271M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43700K .......... .......... .......... .......... .......... 59% 285M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43750K .......... .......... .......... .......... .......... 59% 250M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43800K .......... .......... .......... .......... .......... 59% 288M 0s 2019-12-08T13:07:47.330 INFO:teuthology.orchestra.run.smithi129.stderr: 43850K .......... .......... .......... .......... .......... 59% 254M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 43900K .......... .......... .......... .......... .......... 59% 254M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 43950K .......... .......... .......... .......... .......... 59% 202M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 44000K .......... .......... .......... .......... .......... 59% 258M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 44050K .......... .......... .......... .......... .......... 59% 245M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 44100K .......... .......... .......... .......... .......... 60% 273M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 44150K .......... .......... .......... .......... .......... 60% 230M 0s 2019-12-08T13:07:47.331 INFO:teuthology.orchestra.run.smithi129.stderr: 44200K .......... .......... .......... .......... .......... 60% 256M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44250K .......... .......... .......... .......... .......... 60% 270M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44300K .......... .......... .......... .......... .......... 60% 273M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44350K .......... .......... .......... .......... .......... 60% 211M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44400K .......... .......... .......... .......... .......... 60% 264M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44450K .......... .......... .......... .......... .......... 60% 255M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44500K .......... .......... .......... .......... .......... 60% 250M 0s 2019-12-08T13:07:47.332 INFO:teuthology.orchestra.run.smithi129.stderr: 44550K .......... .......... .......... .......... .......... 60% 228M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44600K .......... .......... .......... .......... .......... 60% 259M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44650K .......... .......... .......... .......... .......... 60% 295M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44700K .......... .......... .......... .......... .......... 60% 281M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44750K .......... .......... .......... .......... .......... 60% 249M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44800K .......... .......... .......... .......... .......... 60% 295M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44850K .......... .......... .......... .......... .......... 61% 273M 0s 2019-12-08T13:07:47.333 INFO:teuthology.orchestra.run.smithi129.stderr: 44900K .......... .......... .......... .......... .......... 61% 298M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 44950K .......... .......... .......... .......... .......... 61% 262M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45000K .......... .......... .......... .......... .......... 61% 298M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45050K .......... .......... .......... .......... .......... 61% 284M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45100K .......... .......... .......... .......... .......... 61% 284M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45150K .......... .......... .......... .......... .......... 61% 238M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45200K .......... .......... .......... .......... .......... 61% 270M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45250K .......... .......... .......... .......... .......... 61% 291M 0s 2019-12-08T13:07:47.334 INFO:teuthology.orchestra.run.smithi129.stderr: 45300K .......... .......... .......... .......... .......... 61% 284M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45350K .......... .......... .......... .......... .......... 61% 250M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45400K .......... .......... .......... .......... .......... 61% 299M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45450K .......... .......... .......... .......... .......... 61% 279M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45500K .......... .......... .......... .......... .......... 61% 271M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45550K .......... .......... .......... .......... .......... 61% 214M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45600K .......... .......... .......... .......... .......... 62% 261M 0s 2019-12-08T13:07:47.335 INFO:teuthology.orchestra.run.smithi129.stderr: 45650K .......... .......... .......... .......... .......... 62% 261M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 45700K .......... .......... .......... .......... .......... 62% 268M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 45750K .......... .......... .......... .......... .......... 62% 244M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 45800K .......... .......... .......... .......... .......... 62% 276M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 45850K .......... .......... .......... .......... .......... 62% 255M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 45900K .......... .......... .......... .......... .......... 62% 263M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 45950K .......... .......... .......... .......... .......... 62% 216M 0s 2019-12-08T13:07:47.336 INFO:teuthology.orchestra.run.smithi129.stderr: 46000K .......... .......... .......... .......... .......... 62% 281M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46050K .......... .......... .......... .......... .......... 62% 275M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46100K .......... .......... .......... .......... .......... 62% 255M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46150K .......... .......... .......... .......... .......... 62% 249M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46200K .......... .......... .......... .......... .......... 62% 257M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46250K .......... .......... .......... .......... .......... 62% 246M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46300K .......... .......... .......... .......... .......... 63% 243M 0s 2019-12-08T13:07:47.337 INFO:teuthology.orchestra.run.smithi129.stderr: 46350K .......... .......... .......... .......... .......... 63% 199M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46400K .......... .......... .......... .......... .......... 63% 266M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46450K .......... .......... .......... .......... .......... 63% 241M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46500K .......... .......... .......... .......... .......... 63% 244M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46550K .......... .......... .......... .......... .......... 63% 233M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46600K .......... .......... .......... .......... .......... 63% 265M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46650K .......... .......... .......... .......... .......... 63% 270M 0s 2019-12-08T13:07:47.338 INFO:teuthology.orchestra.run.smithi129.stderr: 46700K .......... .......... .......... .......... .......... 63% 254M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 46750K .......... .......... .......... .......... .......... 63% 220M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 46800K .......... .......... .......... .......... .......... 63% 326M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 46850K .......... .......... .......... .......... .......... 63% 388M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 46900K .......... .......... .......... .......... .......... 63% 398M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 46950K .......... .......... .......... .......... .......... 63% 376M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 47000K .......... .......... .......... .......... .......... 63% 404M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 47050K .......... .......... .......... .......... .......... 64% 414M 0s 2019-12-08T13:07:47.339 INFO:teuthology.orchestra.run.smithi129.stderr: 47100K .......... .......... .......... .......... .......... 64% 385M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47150K .......... .......... .......... .......... .......... 64% 300M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47200K .......... .......... .......... .......... .......... 64% 354M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47250K .......... .......... .......... .......... .......... 64% 363M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47300K .......... .......... .......... .......... .......... 64% 332M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47350K .......... .......... .......... .......... .......... 64% 276M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47400K .......... .......... .......... .......... .......... 64% 307M 0s 2019-12-08T13:07:47.340 INFO:teuthology.orchestra.run.smithi129.stderr: 47450K .......... .......... .......... .......... .......... 64% 304M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47500K .......... .......... .......... .......... .......... 64% 327M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47550K .......... .......... .......... .......... .......... 64% 270M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47600K .......... .......... .......... .......... .......... 64% 315M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47650K .......... .......... .......... .......... .......... 64% 320M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47700K .......... .......... .......... .......... .......... 64% 322M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47750K .......... .......... .......... .......... .......... 64% 276M 0s 2019-12-08T13:07:47.341 INFO:teuthology.orchestra.run.smithi129.stderr: 47800K .......... .......... .......... .......... .......... 65% 297M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 47850K .......... .......... .......... .......... .......... 65% 334M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 47900K .......... .......... .......... .......... .......... 65% 314M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 47950K .......... .......... .......... .......... .......... 65% 277M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 48000K .......... .......... .......... .......... .......... 65% 324M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 48050K .......... .......... .......... .......... .......... 65% 332M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 48100K .......... .......... .......... .......... .......... 65% 307M 0s 2019-12-08T13:07:47.342 INFO:teuthology.orchestra.run.smithi129.stderr: 48150K .......... .......... .......... .......... .......... 65% 303M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48200K .......... .......... .......... .......... .......... 65% 310M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48250K .......... .......... .......... .......... .......... 65% 334M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48300K .......... .......... .......... .......... .......... 65% 330M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48350K .......... .......... .......... .......... .......... 65% 284M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48400K .......... .......... .......... .......... .......... 65% 341M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48450K .......... .......... .......... .......... .......... 65% 343M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48500K .......... .......... .......... .......... .......... 66% 312M 0s 2019-12-08T13:07:47.343 INFO:teuthology.orchestra.run.smithi129.stderr: 48550K .......... .......... .......... .......... .......... 66% 307M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48600K .......... .......... .......... .......... .......... 66% 332M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48650K .......... .......... .......... .......... .......... 66% 370M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48700K .......... .......... .......... .......... .......... 66% 316M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48750K .......... .......... .......... .......... .......... 66% 268M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48800K .......... .......... .......... .......... .......... 66% 337M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48850K .......... .......... .......... .......... .......... 66% 320M 0s 2019-12-08T13:07:47.344 INFO:teuthology.orchestra.run.smithi129.stderr: 48900K .......... .......... .......... .......... .......... 66% 350M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 48950K .......... .......... .......... .......... .......... 66% 284M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49000K .......... .......... .......... .......... .......... 66% 361M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49050K .......... .......... .......... .......... .......... 66% 378M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49100K .......... .......... .......... .......... .......... 66% 342M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49150K .......... .......... .......... .......... .......... 66% 304M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49200K .......... .......... .......... .......... .......... 66% 329M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49250K .......... .......... .......... .......... .......... 67% 307M 0s 2019-12-08T13:07:47.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49300K .......... .......... .......... .......... .......... 67% 328M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49350K .......... .......... .......... .......... .......... 67% 294M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49400K .......... .......... .......... .......... .......... 67% 325M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49450K .......... .......... .......... .......... .......... 67% 309M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49500K .......... .......... .......... .......... .......... 67% 326M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49550K .......... .......... .......... .......... .......... 67% 262M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49600K .......... .......... .......... .......... .......... 67% 331M 0s 2019-12-08T13:07:47.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49650K .......... .......... .......... .......... .......... 67% 314M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 49700K .......... .......... .......... .......... .......... 67% 326M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 49750K .......... .......... .......... .......... .......... 67% 275M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 49800K .......... .......... .......... .......... .......... 67% 337M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 49850K .......... .......... .......... .......... .......... 67% 330M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 49900K .......... .......... .......... .......... .......... 67% 353M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 49950K .......... .......... .......... .......... .......... 67% 269M 0s 2019-12-08T13:07:47.347 INFO:teuthology.orchestra.run.smithi129.stderr: 50000K .......... .......... .......... .......... .......... 68% 339M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50050K .......... .......... .......... .......... .......... 68% 309M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50100K .......... .......... .......... .......... .......... 68% 345M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50150K .......... .......... .......... .......... .......... 68% 292M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50200K .......... .......... .......... .......... .......... 68% 345M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50250K .......... .......... .......... .......... .......... 68% 341M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50300K .......... .......... .......... .......... .......... 68% 335M 0s 2019-12-08T13:07:47.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50350K .......... .......... .......... .......... .......... 68% 275M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50400K .......... .......... .......... .......... .......... 68% 307M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50450K .......... .......... .......... .......... .......... 68% 318M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50500K .......... .......... .......... .......... .......... 68% 342M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50550K .......... .......... .......... .......... .......... 68% 278M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50600K .......... .......... .......... .......... .......... 68% 328M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50650K .......... .......... .......... .......... .......... 68% 347M 0s 2019-12-08T13:07:47.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50700K .......... .......... .......... .......... .......... 68% 314M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 50750K .......... .......... .......... .......... .......... 69% 278M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 50800K .......... .......... .......... .......... .......... 69% 331M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 50850K .......... .......... .......... .......... .......... 69% 354M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 50900K .......... .......... .......... .......... .......... 69% 338M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 50950K .......... .......... .......... .......... .......... 69% 297M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 51000K .......... .......... .......... .......... .......... 69% 335M 0s 2019-12-08T13:07:47.350 INFO:teuthology.orchestra.run.smithi129.stderr: 51050K .......... .......... .......... .......... .......... 69% 331M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51100K .......... .......... .......... .......... .......... 69% 347M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51150K .......... .......... .......... .......... .......... 69% 306M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51200K .......... .......... .......... .......... .......... 69% 319M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51250K .......... .......... .......... .......... .......... 69% 343M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51300K .......... .......... .......... .......... .......... 69% 340M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51350K .......... .......... .......... .......... .......... 69% 287M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51400K .......... .......... .......... .......... .......... 69% 300M 0s 2019-12-08T13:07:47.351 INFO:teuthology.orchestra.run.smithi129.stderr: 51450K .......... .......... .......... .......... .......... 70% 350M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51500K .......... .......... .......... .......... .......... 70% 322M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51550K .......... .......... .......... .......... .......... 70% 268M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51600K .......... .......... .......... .......... .......... 70% 307M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51650K .......... .......... .......... .......... .......... 70% 345M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51700K .......... .......... .......... .......... .......... 70% 303M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51750K .......... .......... .......... .......... .......... 70% 298M 0s 2019-12-08T13:07:47.352 INFO:teuthology.orchestra.run.smithi129.stderr: 51800K .......... .......... .......... .......... .......... 70% 326M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 51850K .......... .......... .......... .......... .......... 70% 327M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 51900K .......... .......... .......... .......... .......... 70% 359M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 51950K .......... .......... .......... .......... .......... 70% 346M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 52000K .......... .......... .......... .......... .......... 70% 421M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 52050K .......... .......... .......... .......... .......... 70% 384M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 52100K .......... .......... .......... .......... .......... 70% 416M 0s 2019-12-08T13:07:47.353 INFO:teuthology.orchestra.run.smithi129.stderr: 52150K .......... .......... .......... .......... .......... 70% 362M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52200K .......... .......... .......... .......... .......... 71% 415M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52250K .......... .......... .......... .......... .......... 71% 402M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52300K .......... .......... .......... .......... .......... 71% 418M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52350K .......... .......... .......... .......... .......... 71% 358M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52400K .......... .......... .......... .......... .......... 71% 414M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52450K .......... .......... .......... .......... .......... 71% 401M 0s 2019-12-08T13:07:47.354 INFO:teuthology.orchestra.run.smithi129.stderr: 52500K .......... .......... .......... .......... .......... 71% 404M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52550K .......... .......... .......... .......... .......... 71% 377M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52600K .......... .......... .......... .......... .......... 71% 417M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52650K .......... .......... .......... .......... .......... 71% 395M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52700K .......... .......... .......... .......... .......... 71% 323M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52750K .......... .......... .......... .......... .......... 71% 261M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52800K .......... .......... .......... .......... .......... 71% 335M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52850K .......... .......... .......... .......... .......... 71% 345M 0s 2019-12-08T13:07:47.355 INFO:teuthology.orchestra.run.smithi129.stderr: 52900K .......... .......... .......... .......... .......... 71% 319M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 52950K .......... .......... .......... .......... .......... 72% 295M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 53000K .......... .......... .......... .......... .......... 72% 319M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 53050K .......... .......... .......... .......... .......... 72% 346M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 53100K .......... .......... .......... .......... .......... 72% 307M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 53150K .......... .......... .......... .......... .......... 72% 265M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 53200K .......... .......... .......... .......... .......... 72% 320M 0s 2019-12-08T13:07:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 53250K .......... .......... .......... .......... .......... 72% 337M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53300K .......... .......... .......... .......... .......... 72% 344M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53350K .......... .......... .......... .......... .......... 72% 317M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53400K .......... .......... .......... .......... .......... 72% 355M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53450K .......... .......... .......... .......... .......... 72% 310M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53500K .......... .......... .......... .......... .......... 72% 332M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53550K .......... .......... .......... .......... .......... 72% 306M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53600K .......... .......... .......... .......... .......... 72% 428M 0s 2019-12-08T13:07:47.357 INFO:teuthology.orchestra.run.smithi129.stderr: 53650K .......... .......... .......... .......... .......... 73% 434M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 53700K .......... .......... .......... .......... .......... 73% 431M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 53750K .......... .......... .......... .......... .......... 73% 301M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 53800K .......... .......... .......... .......... .......... 73% 332M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 53850K .......... .......... .......... .......... .......... 73% 333M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 53900K .......... .......... .......... .......... .......... 73% 337M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 53950K .......... .......... .......... .......... .......... 73% 279M 0s 2019-12-08T13:07:47.358 INFO:teuthology.orchestra.run.smithi129.stderr: 54000K .......... .......... .......... .......... .......... 73% 303M 0s 2019-12-08T13:07:47.359 INFO:teuthology.orchestra.run.smithi129.stderr: 54050K .......... .......... .......... .......... .......... 73% 333M 0s 2019-12-08T13:07:47.359 INFO:teuthology.orchestra.run.smithi129.stderr: 54100K .......... .......... .......... .......... .......... 73% 322M 0s 2019-12-08T13:07:47.359 INFO:teuthology.orchestra.run.smithi129.stderr: 54150K .......... .......... .......... .......... .......... 73% 293M 0s 2019-12-08T13:07:47.359 INFO:teuthology.orchestra.run.smithi129.stderr: 54200K .......... .......... .......... .......... .......... 73% 320M 0s 2019-12-08T13:07:47.359 INFO:teuthology.orchestra.run.smithi129.stderr: 54250K .......... .......... .......... .......... .......... 73% 334M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54300K .......... .......... .......... .......... .......... 73% 326M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54350K .......... .......... .......... .......... .......... 73% 273M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54400K .......... .......... .......... .......... .......... 74% 338M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54450K .......... .......... .......... .......... .......... 74% 362M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54500K .......... .......... .......... .......... .......... 74% 338M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54550K .......... .......... .......... .......... .......... 74% 307M 0s 2019-12-08T13:07:47.360 INFO:teuthology.orchestra.run.smithi129.stderr: 54600K .......... .......... .......... .......... .......... 74% 326M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54650K .......... .......... .......... .......... .......... 74% 336M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54700K .......... .......... .......... .......... .......... 74% 333M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54750K .......... .......... .......... .......... .......... 74% 280M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54800K .......... .......... .......... .......... .......... 74% 331M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54850K .......... .......... .......... .......... .......... 74% 350M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54900K .......... .......... .......... .......... .......... 74% 380M 0s 2019-12-08T13:07:47.361 INFO:teuthology.orchestra.run.smithi129.stderr: 54950K .......... .......... .......... .......... .......... 74% 363M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55000K .......... .......... .......... .......... .......... 74% 379M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55050K .......... .......... .......... .......... .......... 74% 419M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55100K .......... .......... .......... .......... .......... 74% 399M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55150K .......... .......... .......... .......... .......... 75% 382M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55200K .......... .......... .......... .......... .......... 75% 388M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55250K .......... .......... .......... .......... .......... 75% 412M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55300K .......... .......... .......... .......... .......... 75% 427M 0s 2019-12-08T13:07:47.362 INFO:teuthology.orchestra.run.smithi129.stderr: 55350K .......... .......... .......... .......... .......... 75% 378M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55400K .......... .......... .......... .......... .......... 75% 432M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55450K .......... .......... .......... .......... .......... 75% 420M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55500K .......... .......... .......... .......... .......... 75% 423M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55550K .......... .......... .......... .......... .......... 75% 353M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55600K .......... .......... .......... .......... .......... 75% 407M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55650K .......... .......... .......... .......... .......... 75% 431M 0s 2019-12-08T13:07:47.363 INFO:teuthology.orchestra.run.smithi129.stderr: 55700K .......... .......... .......... .......... .......... 75% 417M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 55750K .......... .......... .......... .......... .......... 75% 370M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 55800K .......... .......... .......... .......... .......... 75% 401M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 55850K .......... .......... .......... .......... .......... 75% 419M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 55900K .......... .......... .......... .......... .......... 76% 428M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 55950K .......... .......... .......... .......... .......... 76% 372M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 56000K .......... .......... .......... .......... .......... 76% 428M 0s 2019-12-08T13:07:47.364 INFO:teuthology.orchestra.run.smithi129.stderr: 56050K .......... .......... .......... .......... .......... 76% 394M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56100K .......... .......... .......... .......... .......... 76% 437M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56150K .......... .......... .......... .......... .......... 76% 347M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56200K .......... .......... .......... .......... .......... 76% 413M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56250K .......... .......... .......... .......... .......... 76% 415M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56300K .......... .......... .......... .......... .......... 76% 429M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56350K .......... .......... .......... .......... .......... 76% 373M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56400K .......... .......... .......... .......... .......... 76% 434M 0s 2019-12-08T13:07:47.365 INFO:teuthology.orchestra.run.smithi129.stderr: 56450K .......... .......... .......... .......... .......... 76% 412M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56500K .......... .......... .......... .......... .......... 76% 425M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56550K .......... .......... .......... .......... .......... 76% 341M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56600K .......... .......... .......... .......... .......... 77% 423M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56650K .......... .......... .......... .......... .......... 77% 431M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56700K .......... .......... .......... .......... .......... 77% 423M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56750K .......... .......... .......... .......... .......... 77% 385M 0s 2019-12-08T13:07:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: 56800K .......... .......... .......... .......... .......... 77% 344M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 56850K .......... .......... .......... .......... .......... 77% 337M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 56900K .......... .......... .......... .......... .......... 77% 335M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 56950K .......... .......... .......... .......... .......... 77% 291M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 57000K .......... .......... .......... .......... .......... 77% 346M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 57050K .......... .......... .......... .......... .......... 77% 344M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 57100K .......... .......... .......... .......... .......... 77% 347M 0s 2019-12-08T13:07:47.367 INFO:teuthology.orchestra.run.smithi129.stderr: 57150K .......... .......... .......... .......... .......... 77% 272M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57200K .......... .......... .......... .......... .......... 77% 346M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57250K .......... .......... .......... .......... .......... 77% 331M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57300K .......... .......... .......... .......... .......... 77% 324M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57350K .......... .......... .......... .......... .......... 78% 284M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57400K .......... .......... .......... .......... .......... 78% 310M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57450K .......... .......... .......... .......... .......... 78% 326M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57500K .......... .......... .......... .......... .......... 78% 333M 0s 2019-12-08T13:07:47.368 INFO:teuthology.orchestra.run.smithi129.stderr: 57550K .......... .......... .......... .......... .......... 78% 283M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57600K .......... .......... .......... .......... .......... 78% 313M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57650K .......... .......... .......... .......... .......... 78% 316M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57700K .......... .......... .......... .......... .......... 78% 301M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57750K .......... .......... .......... .......... .......... 78% 289M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57800K .......... .......... .......... .......... .......... 78% 296M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57850K .......... .......... .......... .......... .......... 78% 299M 0s 2019-12-08T13:07:47.369 INFO:teuthology.orchestra.run.smithi129.stderr: 57900K .......... .......... .......... .......... .......... 78% 335M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 57950K .......... .......... .......... .......... .......... 78% 282M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58000K .......... .......... .......... .......... .......... 78% 317M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58050K .......... .......... .......... .......... .......... 78% 346M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58100K .......... .......... .......... .......... .......... 79% 321M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58150K .......... .......... .......... .......... .......... 79% 294M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58200K .......... .......... .......... .......... .......... 79% 319M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58250K .......... .......... .......... .......... .......... 79% 324M 0s 2019-12-08T13:07:47.370 INFO:teuthology.orchestra.run.smithi129.stderr: 58300K .......... .......... .......... .......... .......... 79% 331M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58350K .......... .......... .......... .......... .......... 79% 295M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58400K .......... .......... .......... .......... .......... 79% 325M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58450K .......... .......... .......... .......... .......... 79% 310M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58500K .......... .......... .......... .......... .......... 79% 344M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58550K .......... .......... .......... .......... .......... 79% 261M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58600K .......... .......... .......... .......... .......... 79% 335M 0s 2019-12-08T13:07:47.371 INFO:teuthology.orchestra.run.smithi129.stderr: 58650K .......... .......... .......... .......... .......... 79% 362M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 58700K .......... .......... .......... .......... .......... 79% 365M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 58750K .......... .......... .......... .......... .......... 79% 321M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 58800K .......... .......... .......... .......... .......... 80% 384M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 58850K .......... .......... .......... .......... .......... 80% 346M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 58900K .......... .......... .......... .......... .......... 80% 330M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 58950K .......... .......... .......... .......... .......... 80% 299M 0s 2019-12-08T13:07:47.372 INFO:teuthology.orchestra.run.smithi129.stderr: 59000K .......... .......... .......... .......... .......... 80% 319M 0s 2019-12-08T13:07:47.373 INFO:teuthology.orchestra.run.smithi129.stderr: 59050K .......... .......... .......... .......... .......... 80% 318M 0s 2019-12-08T13:07:47.373 INFO:teuthology.orchestra.run.smithi129.stderr: 59100K .......... .......... .......... .......... .......... 80% 317M 0s 2019-12-08T13:07:47.373 INFO:teuthology.orchestra.run.smithi129.stderr: 59150K .......... .......... .......... .......... .......... 80% 266M 0s 2019-12-08T13:07:47.373 INFO:teuthology.orchestra.run.smithi129.stderr: 59200K .......... .......... .......... .......... .......... 80% 339M 0s 2019-12-08T13:07:47.374 INFO:teuthology.orchestra.run.smithi129.stderr: 59250K .......... .......... .......... .......... .......... 80% 317M 0s 2019-12-08T13:07:47.374 INFO:teuthology.orchestra.run.smithi129.stderr: 59300K .......... .......... .......... .......... .......... 80% 314M 0s 2019-12-08T13:07:47.374 INFO:teuthology.orchestra.run.smithi129.stderr: 59350K .......... .......... .......... .......... .......... 80% 284M 0s 2019-12-08T13:07:47.374 INFO:teuthology.orchestra.run.smithi129.stderr: 59400K .......... .......... .......... .......... .......... 80% 318M 0s 2019-12-08T13:07:47.374 INFO:teuthology.orchestra.run.smithi129.stderr: 59450K .......... .......... .......... .......... .......... 80% 324M 0s 2019-12-08T13:07:47.374 INFO:teuthology.orchestra.run.smithi129.stderr: 59500K .......... .......... .......... .......... .......... 80% 346M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59550K .......... .......... .......... .......... .......... 81% 284M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59600K .......... .......... .......... .......... .......... 81% 327M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59650K .......... .......... .......... .......... .......... 81% 339M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59700K .......... .......... .......... .......... .......... 81% 332M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59750K .......... .......... .......... .......... .......... 81% 279M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59800K .......... .......... .......... .......... .......... 81% 322M 0s 2019-12-08T13:07:47.375 INFO:teuthology.orchestra.run.smithi129.stderr: 59850K .......... .......... .......... .......... .......... 81% 342M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 59900K .......... .......... .......... .......... .......... 81% 336M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 59950K .......... .......... .......... .......... .......... 81% 279M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 60000K .......... .......... .......... .......... .......... 81% 340M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 60050K .......... .......... .......... .......... .......... 81% 340M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 60100K .......... .......... .......... .......... .......... 81% 331M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 60150K .......... .......... .......... .......... .......... 81% 304M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 60200K .......... .......... .......... .......... .......... 81% 341M 0s 2019-12-08T13:07:47.376 INFO:teuthology.orchestra.run.smithi129.stderr: 60250K .......... .......... .......... .......... .......... 81% 328M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60300K .......... .......... .......... .......... .......... 82% 336M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60350K .......... .......... .......... .......... .......... 82% 295M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60400K .......... .......... .......... .......... .......... 82% 314M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60450K .......... .......... .......... .......... .......... 82% 350M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60500K .......... .......... .......... .......... .......... 82% 371M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60550K .......... .......... .......... .......... .......... 82% 300M 0s 2019-12-08T13:07:47.377 INFO:teuthology.orchestra.run.smithi129.stderr: 60600K .......... .......... .......... .......... .......... 82% 329M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60650K .......... .......... .......... .......... .......... 82% 349M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60700K .......... .......... .......... .......... .......... 82% 342M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60750K .......... .......... .......... .......... .......... 82% 279M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60800K .......... .......... .......... .......... .......... 82% 330M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60850K .......... .......... .......... .......... .......... 82% 330M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60900K .......... .......... .......... .......... .......... 82% 323M 0s 2019-12-08T13:07:47.378 INFO:teuthology.orchestra.run.smithi129.stderr: 60950K .......... .......... .......... .......... .......... 82% 294M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61000K .......... .......... .......... .......... .......... 82% 319M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61050K .......... .......... .......... .......... .......... 83% 327M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61100K .......... .......... .......... .......... .......... 83% 364M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61150K .......... .......... .......... .......... .......... 83% 314M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61200K .......... .......... .......... .......... .......... 83% 336M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61250K .......... .......... .......... .......... .......... 83% 342M 0s 2019-12-08T13:07:47.379 INFO:teuthology.orchestra.run.smithi129.stderr: 61300K .......... .......... .......... .......... .......... 83% 333M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61350K .......... .......... .......... .......... .......... 83% 274M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61400K .......... .......... .......... .......... .......... 83% 333M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61450K .......... .......... .......... .......... .......... 83% 339M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61500K .......... .......... .......... .......... .......... 83% 343M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61550K .......... .......... .......... .......... .......... 83% 300M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61600K .......... .......... .......... .......... .......... 83% 343M 0s 2019-12-08T13:07:47.380 INFO:teuthology.orchestra.run.smithi129.stderr: 61650K .......... .......... .......... .......... .......... 83% 340M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 61700K .......... .......... .......... .......... .......... 83% 335M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 61750K .......... .......... .......... .......... .......... 84% 282M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 61800K .......... .......... .......... .......... .......... 84% 349M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 61850K .......... .......... .......... .......... .......... 84% 346M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 61900K .......... .......... .......... .......... .......... 84% 324M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 61950K .......... .......... .......... .......... .......... 84% 320M 0s 2019-12-08T13:07:47.381 INFO:teuthology.orchestra.run.smithi129.stderr: 62000K .......... .......... .......... .......... .......... 84% 350M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62050K .......... .......... .......... .......... .......... 84% 331M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62100K .......... .......... .......... .......... .......... 84% 338M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62150K .......... .......... .......... .......... .......... 84% 295M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62200K .......... .......... .......... .......... .......... 84% 356M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62250K .......... .......... .......... .......... .......... 84% 328M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62300K .......... .......... .......... .......... .......... 84% 343M 0s 2019-12-08T13:07:47.382 INFO:teuthology.orchestra.run.smithi129.stderr: 62350K .......... .......... .......... .......... .......... 84% 294M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62400K .......... .......... .......... .......... .......... 84% 326M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62450K .......... .......... .......... .......... .......... 84% 323M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62500K .......... .......... .......... .......... .......... 85% 344M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62550K .......... .......... .......... .......... .......... 85% 284M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62600K .......... .......... .......... .......... .......... 85% 346M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62650K .......... .......... .......... .......... .......... 85% 364M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62700K .......... .......... .......... .......... .......... 85% 329M 0s 2019-12-08T13:07:47.383 INFO:teuthology.orchestra.run.smithi129.stderr: 62750K .......... .......... .......... .......... .......... 85% 309M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 62800K .......... .......... .......... .......... .......... 85% 317M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 62850K .......... .......... .......... .......... .......... 85% 342M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 62900K .......... .......... .......... .......... .......... 85% 356M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 62950K .......... .......... .......... .......... .......... 85% 302M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 63000K .......... .......... .......... .......... .......... 85% 340M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 63050K .......... .......... .......... .......... .......... 85% 288M 0s 2019-12-08T13:07:47.384 INFO:teuthology.orchestra.run.smithi129.stderr: 63100K .......... .......... .......... .......... .......... 85% 316M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63150K .......... .......... .......... .......... .......... 85% 209M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63200K .......... .......... .......... .......... .......... 85% 182M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63250K .......... .......... .......... .......... .......... 86% 181M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63300K .......... .......... .......... .......... .......... 86% 174M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63350K .......... .......... .......... .......... .......... 86% 154M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63400K .......... .......... .......... .......... .......... 86% 176M 0s 2019-12-08T13:07:47.385 INFO:teuthology.orchestra.run.smithi129.stderr: 63450K .......... .......... .......... .......... .......... 86% 171M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63500K .......... .......... .......... .......... .......... 86% 180M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63550K .......... .......... .......... .......... .......... 86% 147M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63600K .......... .......... .......... .......... .......... 86% 198M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63650K .......... .......... .......... .......... .......... 86% 302M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63700K .......... .......... .......... .......... .......... 86% 311M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63750K .......... .......... .......... .......... .......... 86% 242M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63800K .......... .......... .......... .......... .......... 86% 328M 0s 2019-12-08T13:07:47.386 INFO:teuthology.orchestra.run.smithi129.stderr: 63850K .......... .......... .......... .......... .......... 86% 332M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 63900K .......... .......... .......... .......... .......... 86% 310M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 63950K .......... .......... .......... .......... .......... 87% 282M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 64000K .......... .......... .......... .......... .......... 87% 325M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 64050K .......... .......... .......... .......... .......... 87% 331M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 64100K .......... .......... .......... .......... .......... 87% 328M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 64150K .......... .......... .......... .......... .......... 87% 297M 0s 2019-12-08T13:07:47.387 INFO:teuthology.orchestra.run.smithi129.stderr: 64200K .......... .......... .......... .......... .......... 87% 332M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64250K .......... .......... .......... .......... .......... 87% 335M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64300K .......... .......... .......... .......... .......... 87% 322M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64350K .......... .......... .......... .......... .......... 87% 265M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64400K .......... .......... .......... .......... .......... 87% 351M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64450K .......... .......... .......... .......... .......... 87% 361M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64500K .......... .......... .......... .......... .......... 87% 351M 0s 2019-12-08T13:07:47.388 INFO:teuthology.orchestra.run.smithi129.stderr: 64550K .......... .......... .......... .......... .......... 87% 320M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64600K .......... .......... .......... .......... .......... 87% 337M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64650K .......... .......... .......... .......... .......... 87% 312M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64700K .......... .......... .......... .......... .......... 88% 384M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64750K .......... .......... .......... .......... .......... 88% 299M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64800K .......... .......... .......... .......... .......... 88% 347M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64850K .......... .......... .......... .......... .......... 88% 337M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64900K .......... .......... .......... .......... .......... 88% 330M 0s 2019-12-08T13:07:47.389 INFO:teuthology.orchestra.run.smithi129.stderr: 64950K .......... .......... .......... .......... .......... 88% 332M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65000K .......... .......... .......... .......... .......... 88% 329M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65050K .......... .......... .......... .......... .......... 88% 341M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65100K .......... .......... .......... .......... .......... 88% 330M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65150K .......... .......... .......... .......... .......... 88% 230M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65200K .......... .......... .......... .......... .......... 88% 305M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65250K .......... .......... .......... .......... .......... 88% 326M 0s 2019-12-08T13:07:47.390 INFO:teuthology.orchestra.run.smithi129.stderr: 65300K .......... .......... .......... .......... .......... 88% 319M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65350K .......... .......... .......... .......... .......... 88% 290M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65400K .......... .......... .......... .......... .......... 88% 321M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65450K .......... .......... .......... .......... .......... 89% 309M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65500K .......... .......... .......... .......... .......... 89% 318M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65550K .......... .......... .......... .......... .......... 89% 265M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65600K .......... .......... .......... .......... .......... 89% 331M 0s 2019-12-08T13:07:47.391 INFO:teuthology.orchestra.run.smithi129.stderr: 65650K .......... .......... .......... .......... .......... 89% 335M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 65700K .......... .......... .......... .......... .......... 89% 317M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 65750K .......... .......... .......... .......... .......... 89% 287M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 65800K .......... .......... .......... .......... .......... 89% 324M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 65850K .......... .......... .......... .......... .......... 89% 316M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 65900K .......... .......... .......... .......... .......... 89% 302M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 65950K .......... .......... .......... .......... .......... 89% 278M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 66000K .......... .......... .......... .......... .......... 89% 332M 0s 2019-12-08T13:07:47.392 INFO:teuthology.orchestra.run.smithi129.stderr: 66050K .......... .......... .......... .......... .......... 89% 330M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66100K .......... .......... .......... .......... .......... 89% 309M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66150K .......... .......... .......... .......... .......... 89% 289M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66200K .......... .......... .......... .......... .......... 90% 315M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66250K .......... .......... .......... .......... .......... 90% 324M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66300K .......... .......... .......... .......... .......... 90% 328M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66350K .......... .......... .......... .......... .......... 90% 282M 0s 2019-12-08T13:07:47.393 INFO:teuthology.orchestra.run.smithi129.stderr: 66400K .......... .......... .......... .......... .......... 90% 325M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66450K .......... .......... .......... .......... .......... 90% 324M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66500K .......... .......... .......... .......... .......... 90% 337M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66550K .......... .......... .......... .......... .......... 90% 286M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66600K .......... .......... .......... .......... .......... 90% 335M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66650K .......... .......... .......... .......... .......... 90% 349M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66700K .......... .......... .......... .......... .......... 90% 331M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66750K .......... .......... .......... .......... .......... 90% 290M 0s 2019-12-08T13:07:47.394 INFO:teuthology.orchestra.run.smithi129.stderr: 66800K .......... .......... .......... .......... .......... 90% 341M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 66850K .......... .......... .......... .......... .......... 90% 350M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 66900K .......... .......... .......... .......... .......... 91% 297M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 66950K .......... .......... .......... .......... .......... 91% 209M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 67000K .......... .......... .......... .......... .......... 91% 310M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 67050K .......... .......... .......... .......... .......... 91% 332M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 67100K .......... .......... .......... .......... .......... 91% 333M 0s 2019-12-08T13:07:47.395 INFO:teuthology.orchestra.run.smithi129.stderr: 67150K .......... .......... .......... .......... .......... 91% 265M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67200K .......... .......... .......... .......... .......... 91% 319M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67250K .......... .......... .......... .......... .......... 91% 319M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67300K .......... .......... .......... .......... .......... 91% 337M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67350K .......... .......... .......... .......... .......... 91% 304M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67400K .......... .......... .......... .......... .......... 91% 342M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67450K .......... .......... .......... .......... .......... 91% 328M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67500K .......... .......... .......... .......... .......... 91% 330M 0s 2019-12-08T13:07:47.396 INFO:teuthology.orchestra.run.smithi129.stderr: 67550K .......... .......... .......... .......... .......... 91% 258M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67600K .......... .......... .......... .......... .......... 91% 328M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67650K .......... .......... .......... .......... .......... 92% 321M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67700K .......... .......... .......... .......... .......... 92% 347M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67750K .......... .......... .......... .......... .......... 92% 298M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67800K .......... .......... .......... .......... .......... 92% 322M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67850K .......... .......... .......... .......... .......... 92% 313M 0s 2019-12-08T13:07:47.397 INFO:teuthology.orchestra.run.smithi129.stderr: 67900K .......... .......... .......... .......... .......... 92% 377M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 67950K .......... .......... .......... .......... .......... 92% 277M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 68000K .......... .......... .......... .......... .......... 92% 319M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 68050K .......... .......... .......... .......... .......... 92% 311M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 68100K .......... .......... .......... .......... .......... 92% 304M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 68150K .......... .......... .......... .......... .......... 92% 269M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 68200K .......... .......... .......... .......... .......... 92% 319M 0s 2019-12-08T13:07:47.398 INFO:teuthology.orchestra.run.smithi129.stderr: 68250K .......... .......... .......... .......... .......... 92% 302M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68300K .......... .......... .......... .......... .......... 92% 318M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68350K .......... .......... .......... .......... .......... 92% 281M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68400K .......... .......... .......... .......... .......... 93% 313M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68450K .......... .......... .......... .......... .......... 93% 330M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68500K .......... .......... .......... .......... .......... 93% 317M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68550K .......... .......... .......... .......... .......... 93% 293M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68600K .......... .......... .......... .......... .......... 93% 332M 0s 2019-12-08T13:07:47.399 INFO:teuthology.orchestra.run.smithi129.stderr: 68650K .......... .......... .......... .......... .......... 93% 324M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 68700K .......... .......... .......... .......... .......... 93% 326M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 68750K .......... .......... .......... .......... .......... 93% 196M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 68800K .......... .......... .......... .......... .......... 93% 306M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 68850K .......... .......... .......... .......... .......... 93% 310M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 68900K .......... .......... .......... .......... .......... 93% 348M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 68950K .......... .......... .......... .......... .......... 93% 293M 0s 2019-12-08T13:07:47.400 INFO:teuthology.orchestra.run.smithi129.stderr: 69000K .......... .......... .......... .......... .......... 93% 317M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69050K .......... .......... .......... .......... .......... 93% 323M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69100K .......... .......... .......... .......... .......... 94% 333M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69150K .......... .......... .......... .......... .......... 94% 270M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69200K .......... .......... .......... .......... .......... 94% 319M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69250K .......... .......... .......... .......... .......... 94% 325M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69300K .......... .......... .......... .......... .......... 94% 333M 0s 2019-12-08T13:07:47.401 INFO:teuthology.orchestra.run.smithi129.stderr: 69350K .......... .......... .......... .......... .......... 94% 292M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69400K .......... .......... .......... .......... .......... 94% 314M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69450K .......... .......... .......... .......... .......... 94% 320M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69500K .......... .......... .......... .......... .......... 94% 331M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69550K .......... .......... .......... .......... .......... 94% 255M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69600K .......... .......... .......... .......... .......... 94% 354M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69650K .......... .......... .......... .......... .......... 94% 337M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69700K .......... .......... .......... .......... .......... 94% 320M 0s 2019-12-08T13:07:47.402 INFO:teuthology.orchestra.run.smithi129.stderr: 69750K .......... .......... .......... .......... .......... 94% 289M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 69800K .......... .......... .......... .......... .......... 94% 313M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 69850K .......... .......... .......... .......... .......... 95% 334M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 69900K .......... .......... .......... .......... .......... 95% 330M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 69950K .......... .......... .......... .......... .......... 95% 264M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 70000K .......... .......... .......... .......... .......... 95% 343M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 70050K .......... .......... .......... .......... .......... 95% 319M 0s 2019-12-08T13:07:47.403 INFO:teuthology.orchestra.run.smithi129.stderr: 70100K .......... .......... .......... .......... .......... 95% 345M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70150K .......... .......... .......... .......... .......... 95% 288M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70200K .......... .......... .......... .......... .......... 95% 338M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70250K .......... .......... .......... .......... .......... 95% 331M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70300K .......... .......... .......... .......... .......... 95% 350M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70350K .......... .......... .......... .......... .......... 95% 280M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70400K .......... .......... .......... .......... .......... 95% 308M 0s 2019-12-08T13:07:47.404 INFO:teuthology.orchestra.run.smithi129.stderr: 70450K .......... .......... .......... .......... .......... 95% 325M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70500K .......... .......... .......... .......... .......... 95% 317M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70550K .......... .......... .......... .......... .......... 95% 274M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70600K .......... .......... .......... .......... .......... 96% 317M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70650K .......... .......... .......... .......... .......... 96% 333M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70700K .......... .......... .......... .......... .......... 96% 315M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70750K .......... .......... .......... .......... .......... 96% 278M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70800K .......... .......... .......... .......... .......... 96% 382M 0s 2019-12-08T13:07:47.405 INFO:teuthology.orchestra.run.smithi129.stderr: 70850K .......... .......... .......... .......... .......... 96% 433M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 70900K .......... .......... .......... .......... .......... 96% 366M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 70950K .......... .......... .......... .......... .......... 96% 357M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 71000K .......... .......... .......... .......... .......... 96% 367M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 71050K .......... .......... .......... .......... .......... 96% 314M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 71100K .......... .......... .......... .......... .......... 96% 315M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 71150K .......... .......... .......... .......... .......... 96% 284M 0s 2019-12-08T13:07:47.406 INFO:teuthology.orchestra.run.smithi129.stderr: 71200K .......... .......... .......... .......... .......... 96% 300M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71250K .......... .......... .......... .......... .......... 96% 306M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71300K .......... .......... .......... .......... .......... 96% 331M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71350K .......... .......... .......... .......... .......... 97% 276M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71400K .......... .......... .......... .......... .......... 97% 339M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71450K .......... .......... .......... .......... .......... 97% 320M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71500K .......... .......... .......... .......... .......... 97% 307M 0s 2019-12-08T13:07:47.407 INFO:teuthology.orchestra.run.smithi129.stderr: 71550K .......... .......... .......... .......... .......... 97% 288M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71600K .......... .......... .......... .......... .......... 97% 350M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71650K .......... .......... .......... .......... .......... 97% 303M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71700K .......... .......... .......... .......... .......... 97% 356M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71750K .......... .......... .......... .......... .......... 97% 293M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71800K .......... .......... .......... .......... .......... 97% 322M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71850K .......... .......... .......... .......... .......... 97% 316M 0s 2019-12-08T13:07:47.408 INFO:teuthology.orchestra.run.smithi129.stderr: 71900K .......... .......... .......... .......... .......... 97% 333M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 71950K .......... .......... .......... .......... .......... 97% 276M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72000K .......... .......... .......... .......... .......... 97% 317M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72050K .......... .......... .......... .......... .......... 98% 282M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72100K .......... .......... .......... .......... .......... 98% 349M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72150K .......... .......... .......... .......... .......... 98% 274M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72200K .......... .......... .......... .......... .......... 98% 312M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72250K .......... .......... .......... .......... .......... 98% 299M 0s 2019-12-08T13:07:47.409 INFO:teuthology.orchestra.run.smithi129.stderr: 72300K .......... .......... .......... .......... .......... 98% 312M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72350K .......... .......... .......... .......... .......... 98% 262M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72400K .......... .......... .......... .......... .......... 98% 316M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72450K .......... .......... .......... .......... .......... 98% 327M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72500K .......... .......... .......... .......... .......... 98% 327M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72550K .......... .......... .......... .......... .......... 98% 271M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72600K .......... .......... .......... .......... .......... 98% 313M 0s 2019-12-08T13:07:47.410 INFO:teuthology.orchestra.run.smithi129.stderr: 72650K .......... .......... .......... .......... .......... 98% 309M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 72700K .......... .......... .......... .......... .......... 98% 326M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 72750K .......... .......... .......... .......... .......... 98% 280M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 72800K .......... .......... .......... .......... .......... 99% 335M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 72850K .......... .......... .......... .......... .......... 99% 334M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 72900K .......... .......... .......... .......... .......... 99% 341M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 72950K .......... .......... .......... .......... .......... 99% 273M 0s 2019-12-08T13:07:47.411 INFO:teuthology.orchestra.run.smithi129.stderr: 73000K .......... .......... .......... .......... .......... 99% 335M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73050K .......... .......... .......... .......... .......... 99% 322M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73100K .......... .......... .......... .......... .......... 99% 351M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73150K .......... .......... .......... .......... .......... 99% 297M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73200K .......... .......... .......... .......... .......... 99% 307M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73250K .......... .......... .......... .......... .......... 99% 324M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73300K .......... .......... .......... .......... .......... 99% 359M 0s 2019-12-08T13:07:47.412 INFO:teuthology.orchestra.run.smithi129.stderr: 73350K .......... .......... .......... .......... .......... 99% 285M 0s 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr: 73400K .......... .......... .......... .......... .......... 99% 360M 0s 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr: 73450K .......... .......... .......... .......... .......... 99% 330M 0s 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr: 73500K .......... .......... .......... .......... .......... 99% 342M 0s 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr: 73550K ........ 100% 316M=0.3s 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08 13:07:46 (258 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-12-08T13:07:47.413 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout:Archive: gradle-4.7-bin.zip 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout: creating: /opt/gradle/gradle-4.7/ 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-12-08T13:07:47.414 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-12-08T13:07:47.415 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-12-08T13:07:47.416 INFO:teuthology.orchestra.run.smithi129.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-12-08T13:07:47.417 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-12-08T13:07:47.418 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-12-08T13:07:47.419 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-12-08T13:07:47.420 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-12-08T13:07:47.421 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-12-08T13:07:47.422 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-12-08T13:07:47.423 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-12-08T13:07:47.424 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-12-08T13:07:47.424 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-12-08T13:07:47.424 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-12-08T13:07:47.424 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-12-08T13:07:47.424 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-12-08T13:07:47.424 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-12-08T13:07:47.425 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-12-08T13:07:47.425 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-12-08T13:07:47.426 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-12-08T13:07:47.426 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-12-08T13:07:47.426 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-12-08T13:07:47.426 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-12-08T13:07:47.426 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-12-08T13:07:47.426 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-12-08T13:07:47.427 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-12-08T13:07:47.428 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-12-08T13:07:47.429 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-12-08T13:07:47.430 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-12-08T13:07:47.431 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-12-08T13:07:47.432 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-12-08T13:07:47.433 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-12-08T13:07:47.434 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-12-08T13:07:47.435 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-12-08T13:07:47.436 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-12-08T13:07:47.437 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-12-08T13:07:47.438 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-12-08T13:07:47.439 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-12-08T13:07:47.440 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-12-08T13:07:47.441 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-12-08T13:07:47.442 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-12-08T13:07:47.443 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-12-08T13:07:47.444 INFO:teuthology.orchestra.run.smithi129.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-12-08T13:07:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout:Welcome to Gradle 4.7! 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout:Here are the highlights of this release: 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: - Incremental annotation processing 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: - JDK 10 support 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: - Grouped non-interactive console logs 2019-12-08T13:07:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: - Failed tests are re-run first for quicker feedback 2019-12-08T13:07:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:47.846 INFO:teuthology.orchestra.run.smithi129.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-12-08T13:07:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:48.209 INFO:teuthology.orchestra.run.smithi129.stdout:------------------------------------------------------------ 2019-12-08T13:07:48.209 INFO:teuthology.orchestra.run.smithi129.stdout:Gradle 4.7 2019-12-08T13:07:48.209 INFO:teuthology.orchestra.run.smithi129.stdout:------------------------------------------------------------ 2019-12-08T13:07:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:48.209 INFO:teuthology.orchestra.run.smithi129.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout:Groovy: 2.4.12 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout:OS: Linux 3.10.0-1062.9.1.el7.x86_64 amd64 2019-12-08T13:07:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:48.213 INFO:teuthology.orchestra.run.smithi129:> sudo keytool -import -alias smithi129.front.sepia.ceph.com -keystore $(readlink -e $(dirname $(readlink -e $(which keytool)))/../lib/security/cacerts) -file /home/ubuntu/cephtest/ca/rgw.client.0.crt -storepass changeit 2019-12-08T13:07:48.623 INFO:teuthology.orchestra.run.smithi129.stderr:Certificate was added to keystore 2019-12-08T13:07:48.643 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-12-08T13:07:48.643 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-08T13:07:48.643 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi129.front.sepia.ceph.com 2019-12-08T13:07:48.643 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-08T13:07:48.643 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi181.front.sepia.ceph.com 2019-12-08T13:07:48.643 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-12-08T13:07:49.071 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-12-08T13:07:49.071 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-12-08T13:07:49.072 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'SIUQLZFKUOWAPRZVZJYP', 's3dir': 'folder', 'endpoint': 'smithi129.front.sepia.ceph.com:443', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 443, 'access_secret': 'LDf07ogNPpCbCySssYlGcO/BSwzyBQgNti9GMfcBdj5eaXbiQZawTg==', 'host': 'smithi129.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': True, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-12-08T13:07:49.072 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-12-08T13:07:49.072 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'SIUQLZFKUOWAPRZVZJYP', '--secret', 'LDf07ogNPpCbCySssYlGcO/BSwzyBQgNti9GMfcBdj5eaXbiQZawTg==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-12-08T13:07:49.072 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key SIUQLZFKUOWAPRZVZJYP --secret LDf07ogNPpCbCySssYlGcO/BSwzyBQgNti9GMfcBdj5eaXbiQZawTg== --email tester.client.0_test@test.test --cluster ceph 2019-12-08T13:07:49.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.205+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.228 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.230+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.228 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.230+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.239+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.239+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.242 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.243+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.242 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.243+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.249 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.251+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.434+0000 7f5989f2cec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:07:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.434+0000 7f5989f2cec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:07:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.434+0000 7f5989f2cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:07:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.434+0000 7f5989f2cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:07:49.434 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.434+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.440+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:49.440 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.440+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.447 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.449+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:49.448 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.449+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.464+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.464+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.481+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:49.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.481+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.487+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.487+0000 7f5989f2cec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:07:49.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.487+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.495+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.495+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.503+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:49.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.503+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.519 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.521+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:49.519 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.521+0000 7f5989f2cec0 20 zonegroup default 2019-12-08T13:07:49.519 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.521+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.524 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.526+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.525 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.526+0000 7f5989f2cec0 10 Cannot find current period zone using local zone 2019-12-08T13:07:49.525 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.526+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.531 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.533+0000 7f5989f2cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:49.531 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.533+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.544+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:49.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.544+0000 7f5989f2cec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:49.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.551+0000 7f5989f2cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:49.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.551+0000 7f5989f2cec0 20 zone default found 2019-12-08T13:07:49.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.551+0000 7f5989f2cec0 20 started sync module instance, tier type = 2019-12-08T13:07:49.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.551+0000 7f5989f2cec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:07:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=0 2019-12-08T13:07:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=1 2019-12-08T13:07:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=2 2019-12-08T13:07:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=3 2019-12-08T13:07:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=4 2019-12-08T13:07:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=5 2019-12-08T13:07:49.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:49.998+0000 7f5989f2cec0 20 add_watcher() i=6 2019-12-08T13:07:50.027 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:50.028+0000 7f5989f2cec0 20 add_watcher() i=7 2019-12-08T13:07:50.027 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:50.028+0000 7f5989f2cec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:07:50.027 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:50.029+0000 7f5947fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:07:50.033 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:50.035+0000 7f5976955700 0 -- 172.21.15.129:0/2132472099 >> v2:172.21.15.129:3300/0 conn(0x5649f8338c50 msgr2=0x5649f833b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T13:07:50.236 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:50.237+0000 7f5976955700 0 --2- 172.21.15.129:0/2132472099 >> v2:172.21.15.129:3300/0 conn(0x5649f8338c50 0x5649f833b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-08T13:07:50.266 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:50.268+0000 7f5977957700 0 -- 172.21.15.129:0/1843269575 >> v2:172.21.15.129:3300/0 conn(0x5649f8338b30 msgr2=0x5649f8338f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:07:51.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.698+0000 7f58b67fc700 20 reqs_thread_entry: start 2019-12-08T13:07:51.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.698+0000 7f58affff700 20 reqs_thread_entry: start 2019-12-08T13:07:51.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.698+0000 7f5989f2cec0 20 init_complete bucket index max shards: 0 2019-12-08T13:07:51.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.699+0000 7f58ae7fc700 20 reqs_thread_entry: start 2019-12-08T13:07:51.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.699+0000 7f5989f2cec0 20 get_system_obj_state: rctx=0x7ffc00eb4f88 obj=default.rgw.meta:users.uid:tester.client.0 state=0x5649f83b7c50 s->prefetch_data=0 2019-12-08T13:07:51.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.699+0000 7f5989f2cec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-12-08T13:07:51.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.704+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-12-08T13:07:51.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.704+0000 7f5989f2cec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-08T13:07:51.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.704+0000 7f5989f2cec0 20 get_system_obj_state: rctx=0x7ffc00eb4f98 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x5649f83b99e0 s->prefetch_data=0 2019-12-08T13:07:51.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.704+0000 7f5989f2cec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-12-08T13:07:51.707 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.709+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-12-08T13:07:51.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.709+0000 7f5989f2cec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-08T13:07:51.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.709+0000 7f5989f2cec0 20 get_system_obj_state: rctx=0x7ffc00eb4f98 obj=default.rgw.meta:users.keys:SIUQLZFKUOWAPRZVZJYP state=0x5649f83b99e0 s->prefetch_data=0 2019-12-08T13:07:51.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.709+0000 7f5989f2cec0 10 cache get: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP : miss 2019-12-08T13:07:51.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.714+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP info.flags=0x0 2019-12-08T13:07:51.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.714+0000 7f5989f2cec0 10 adding default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP to cache LRU end 2019-12-08T13:07:51.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.714+0000 7f5989f2cec0 20 get_system_obj_state: rctx=0x7ffc00eb4ba8 obj=default.rgw.meta:users.keys:SIUQLZFKUOWAPRZVZJYP state=0x5649f83b99e0 s->prefetch_data=0 2019-12-08T13:07:51.713 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.714+0000 7f5989f2cec0 10 cache get: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP : type miss (requested=0x6, cached=0x0) 2019-12-08T13:07:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.718+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP info.flags=0x0 2019-12-08T13:07:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.718+0000 7f5989f2cec0 10 moving default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP to cache LRU end 2019-12-08T13:07:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.718+0000 7f5989f2cec0 20 get_system_obj_state: rctx=0x7ffc00eb4d28 obj=default.rgw.meta:users.keys:SIUQLZFKUOWAPRZVZJYP state=0x5649f83b99e0 s->prefetch_data=0 2019-12-08T13:07:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.718+0000 7f5989f2cec0 10 cache get: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP : type miss (requested=0x6, cached=0x0) 2019-12-08T13:07:51.721 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.722+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP info.flags=0x0 2019-12-08T13:07:51.721 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.722+0000 7f5989f2cec0 10 moving default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP to cache LRU end 2019-12-08T13:07:51.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.782+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-08T13:07:51.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.782+0000 7f5989f2cec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-08T13:07:51.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.782+0000 7f5989f2cec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-12-08T13:07:51.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.806+0000 7f595cff9700 10 RGWWatcher::handle_notify() notify_id 562640715778 cookie 94875696631664 notifier 4749 bl.length()=632 2019-12-08T13:07:51.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.806+0000 7f595cff9700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-08T13:07:51.805 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:51.806+0000 7f595cff9700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-08T13:07:52.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.302+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-08T13:07:52.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.303+0000 7f5989f2cec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-08T13:07:52.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.303+0000 7f5989f2cec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-12-08T13:07:52.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.334+0000 7f595cff9700 10 RGWWatcher::handle_notify() notify_id 562640715779 cookie 94875696609232 notifier 4749 bl.length()=200 2019-12-08T13:07:52.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.334+0000 7f595cff9700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-08T13:07:52.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.334+0000 7f595cff9700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-08T13:07:52.699 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.701+0000 7f5989f2cec0 10 cache put: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP info.flags=0x7 2019-12-08T13:07:52.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.701+0000 7f5989f2cec0 10 moving default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP to cache LRU end 2019-12-08T13:07:52.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.701+0000 7f5989f2cec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=189 2019-12-08T13:07:52.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.713+0000 7f595cff9700 10 RGWWatcher::handle_notify() notify_id 562640715781 cookie 94875696626720 notifier 4749 bl.length()=189 2019-12-08T13:07:52.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.713+0000 7f595cff9700 10 cache put: name=default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP info.flags=0x7 2019-12-08T13:07:52.713 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:52.713+0000 7f595cff9700 10 moving default.rgw.meta+users.keys+SIUQLZFKUOWAPRZVZJYP to cache LRU end 2019-12-08T13:07:53.356 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T13:07:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "tester.client.0", 2019-12-08T13:07:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Ms. tester.client.0", 2019-12-08T13:07:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "tester.client.0_test@test.test", 2019-12-08T13:07:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T13:07:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [], 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [ 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "tester.client.0", 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "access_key": "SIUQLZFKUOWAPRZVZJYP", 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "LDf07ogNPpCbCySssYlGcO/BSwzyBQgNti9GMfcBdj5eaXbiQZawTg==" 2019-12-08T13:07:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [], 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T13:07:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T13:07:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T13:07:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T13:07:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T13:07:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T13:07:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T13:07:53.361 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T13:07:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:07:53.429 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.430+0000 7f5989f2cec0 20 remove_watcher() i=0 2019-12-08T13:07:53.429 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.430+0000 7f5989f2cec0 2 removed watcher, disabling cache 2019-12-08T13:07:53.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.476+0000 7f5989f2cec0 20 remove_watcher() i=1 2019-12-08T13:07:53.511 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.513+0000 7f5989f2cec0 20 remove_watcher() i=2 2019-12-08T13:07:53.563 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.565+0000 7f5989f2cec0 20 remove_watcher() i=3 2019-12-08T13:07:53.624 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.626+0000 7f5989f2cec0 20 remove_watcher() i=4 2019-12-08T13:07:53.668 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.670+0000 7f5989f2cec0 20 remove_watcher() i=5 2019-12-08T13:07:53.722 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.724+0000 7f5989f2cec0 20 remove_watcher() i=6 2019-12-08T13:07:53.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:53.772+0000 7f5989f2cec0 20 remove_watcher() i=7 2019-12-08T13:07:53.782 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-12-08T13:07:53.841 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-12-08T13:07:53.842 INFO:teuthology.orchestra.run.smithi129:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-12-08T13:07:53.927 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:07:54.053 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.055+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.076+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.074 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.076+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.085+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.085+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.088 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.090+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.088 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.090+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.096 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.098+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:54.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.283+0000 7f015810fec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:07:54.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.283+0000 7f015810fec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:07:54.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.283+0000 7f015810fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:07:54.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.283+0000 7f015810fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:07:54.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.283+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.300 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.302+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:54.300 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.302+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.312+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:54.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.313+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.322+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:54.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.322+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.339 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.341+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:54.339 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.341+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.346+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.346+0000 7f015810fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:07:54.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.346+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.351+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.351+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.358 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.360+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:54.358 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.360+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.377+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:07:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.377+0000 7f015810fec0 20 zonegroup default 2019-12-08T13:07:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.377+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.382+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.382+0000 7f015810fec0 10 Cannot find current period zone using local zone 2019-12-08T13:07:54.381 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.382+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.387+0000 7f015810fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:07:54.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.387+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.394+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:07:54.393 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.394+0000 7f015810fec0 20 rados->read ofs=0 len=0 2019-12-08T13:07:54.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.401+0000 7f015810fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:07:54.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.401+0000 7f015810fec0 20 zone default found 2019-12-08T13:07:54.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.401+0000 7f015810fec0 20 started sync module instance, tier type = 2019-12-08T13:07:54.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.401+0000 7f015810fec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:07:54.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=0 2019-12-08T13:07:54.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=1 2019-12-08T13:07:54.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=2 2019-12-08T13:07:54.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=3 2019-12-08T13:07:54.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=4 2019-12-08T13:07:54.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=5 2019-12-08T13:07:54.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.849+0000 7f015810fec0 20 add_watcher() i=6 2019-12-08T13:07:54.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.883+0000 7f015810fec0 20 add_watcher() i=7 2019-12-08T13:07:54.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.883+0000 7f015810fec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:07:54.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.884+0000 7f0114fe9700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:07:54.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.888+0000 7f0144b38700 0 -- >> v2:172.21.15.181:3300/0 conn(0x55f3ee239be0 msgr2=0x55f3ee262220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T13:07:54.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:54.963+0000 7f0145b3a700 0 -- 172.21.15.129:0/1184440722 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7f0088022480 msgr2=0x7f0088024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:07:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.411+0000 7f0086ffd700 20 reqs_thread_entry: start 2019-12-08T13:07:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.411+0000 7f0084ff9700 20 reqs_thread_entry: start 2019-12-08T13:07:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.411+0000 7f015810fec0 20 init_complete bucket index max shards: 0 2019-12-08T13:07:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.411+0000 7f0072ffd700 20 reqs_thread_entry: start 2019-12-08T13:07:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.411+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:07:56.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.481+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:56.497 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.499+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:56.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.509+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:56.558 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.560+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:07:56.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.603+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:56.616 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.618+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:56.646 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T13:07:56.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_OmEdPPC3Kc2Wq1ggqZxAEeQRu2MSvtX', time=2019-12-08T15:07:23.797439+0000, chain.objs.size()=1 2019-12-08T13:07:56.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.cIl9cTf4VMs-1H8RsN2gDR8UTeQZsUV_0 2019-12-08T13:07:56.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.84', time=2019-12-08T15:07:29.928663+0000, chain.objs.size()=3 2019-12-08T13:07:56.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.6__shadow_.DviLCFavuAA1BpJbT2ur7dRhh2y-rco_0 2019-12-08T13:07:56.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.6__shadow_.DviLCFavuAA1BpJbT2ur7dRhh2y-rco_1 2019-12-08T13:07:56.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.648+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.6__shadow_.DviLCFavuAA1BpJbT2ur7dRhh2y-rco_2 2019-12-08T13:07:56.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.875+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:56.994 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:56.996+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:07:57.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.068+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:57.100 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.102+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:57.151 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.152+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:07:57.151 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.152+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.210', time=2019-12-08T15:07:28.999690+0000, chain.objs.size()=3 2019-12-08T13:07:57.151 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.152+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-STANDARD.2~5hxBOxv0dpuyMd0i4GQWc6Kde_j2pdP.1 2019-12-08T13:07:57.151 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.152+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-STANDARD.2~5hxBOxv0dpuyMd0i4GQWc6Kde_j2pdP.2 2019-12-08T13:07:57.151 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.152+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-STANDARD.2~5hxBOxv0dpuyMd0i4GQWc6Kde_j2pdP.3 2019-12-08T13:07:57.280 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.282+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:57.397 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.399+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:07:57.440 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.442+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:57.462 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.463+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-08T13:07:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.112', time=2019-12-08T15:07:22.559681+0000, chain.objs.size()=2 2019-12-08T13:07:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.amf47Jttp1tC3GMNuMjIBVqi-T21M8x_0 2019-12-08T13:07:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.amf47Jttp1tC3GMNuMjIBVqi-T21M8x_1 2019-12-08T13:07:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.114', time=2019-12-08T15:07:22.936893+0000, chain.objs.size()=2 2019-12-08T13:07:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.amf47Jttp1tC3GMNuMjIBVqi-T21M8x_0 2019-12-08T13:07:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.amf47Jttp1tC3GMNuMjIBVqi-T21M8x_1 2019-12-08T13:07:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_QUm2ng25LM-V7OCkKUsPz-jl8KOL4fH', time=2019-12-08T15:07:23.236061+0000, chain.objs.size()=1 2019-12-08T13:07:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.494+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.1CxR2APbZUFRWCqk718CKyUys-LTz33_1 2019-12-08T13:07:57.715 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.717+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:57.857 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.858+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:07:57.930 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.932+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.959+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:57.967 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:57.968+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:58.010 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.011+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:07:58.086 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.087+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:58.122 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.124+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.186+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:07:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.186+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.242', time=2019-12-08T15:07:26.963721+0000, chain.objs.size()=3 2019-12-08T13:07:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.186+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-STANDARD.2~cUeluRY4r_xW6o_PPNlg3eUS1b0PHGP.1 2019-12-08T13:07:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.186+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-STANDARD.2~cUeluRY4r_xW6o_PPNlg3eUS1b0PHGP.2 2019-12-08T13:07:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.186+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-STANDARD.2~cUeluRY4r_xW6o_PPNlg3eUS1b0PHGP.3 2019-12-08T13:07:58.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.404+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.696+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:07:58.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.739+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:58.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.754+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:58.761 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.762+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:58.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.798+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:07:58.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.861+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:58.878 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.879+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:58.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.905+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:07:58.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.905+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_MEfm5tDRhsEbmHObP2aF3toswE_TVua', time=2019-12-08T15:07:25.636281+0000, chain.objs.size()=3 2019-12-08T13:07:58.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.905+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.CyFMl_u_i97Q3EbJl-dFnXUONTnE-se_0 2019-12-08T13:07:58.905 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.905+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.CyFMl_u_i97Q3EbJl-dFnXUONTnE-se_1 2019-12-08T13:07:58.905 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:58.905+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.CyFMl_u_i97Q3EbJl-dFnXUONTnE-se_2 2019-12-08T13:07:59.115 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.116+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:59.233 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.234+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:07:59.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.291+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:59.312 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.313+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:59.322 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.324+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:07:59.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.324+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.109', time=2019-12-08T15:07:21.926963+0000, chain.objs.size()=2 2019-12-08T13:07:59.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.324+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.pNqwMP_b10v9jR5Pe91phu_0XAO4YQ5_0 2019-12-08T13:07:59.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.324+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.pNqwMP_b10v9jR5Pe91phu_0XAO4YQ5_1 2019-12-08T13:07:59.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.368+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:59.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.465+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:07:59.513 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.514+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:59.527 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.529+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:59.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.543+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:07:59.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.543+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.55', time=2019-12-08T15:07:18.076458+0000, chain.objs.size()=4 2019-12-08T13:07:59.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.543+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.4__multipart_foo.2~hLXbbHQS-rcWjnpE6MMqkQ6vGgrhsMN.1 2019-12-08T13:07:59.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.543+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.4__multipart_foo.2~hLXbbHQS-rcWjnpE6MMqkQ6vGgrhsMN.2 2019-12-08T13:07:59.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.543+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.4__multipart_foo.2~hLXbbHQS-rcWjnpE6MMqkQ6vGgrhsMN.3 2019-12-08T13:07:59.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.543+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.4__multipart_foo.2~hLXbbHQS-rcWjnpE6MMqkQ6vGgrhsMN.4 2019-12-08T13:07:59.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.666+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:07:59.771 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.772+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:07:59.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.827+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:59.853 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.855+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:59.872 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.873+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:59.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.916+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:07:59.973 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.974+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:07:59.989 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.991+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:07:59.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:07:59.999+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:00.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.037+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:08:00.118 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.119+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:00.151 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.152+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:00.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-08T13:08:00.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_XMG4bVH-1iv1BRb4wBjFAllbeCSrKc3', time=2019-12-08T15:07:25.874559+0000, chain.objs.size()=2 2019-12-08T13:08:00.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.pcMyWqqEHsnpsTWJtvFrFAilBCW10Tq_1 2019-12-08T13:08:00.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.pcMyWqqEHsnpsTWJtvFrFAilBCW10Tq_2 2019-12-08T13:08:00.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_cLyXgqqAoeyGK6RIbQ4ksYdjQyzxikO', time=2019-12-08T15:07:27.344891+0000, chain.objs.size()=1 2019-12-08T13:08:00.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.z7LYOnefFYF8_PwjuqFDFF9qlCRQzXi_0 2019-12-08T13:08:00.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.203', time=2019-12-08T15:07:28.769286+0000, chain.objs.size()=5 2019-12-08T13:08:00.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-LUKEWARM.2~3zJf7JWf6kzDKtKHQ5c9W5awDeQlj8C.1 2019-12-08T13:08:00.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.8__shadow_foo-LUKEWARM.2~3zJf7JWf6kzDKtKHQ5c9W5awDeQlj8C.1_1 2019-12-08T13:08:00.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-LUKEWARM.2~3zJf7JWf6kzDKtKHQ5c9W5awDeQlj8C.2 2019-12-08T13:08:00.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.8__shadow_foo-LUKEWARM.2~3zJf7JWf6kzDKtKHQ5c9W5awDeQlj8C.2_1 2019-12-08T13:08:00.204 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.203+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-LUKEWARM.2~3zJf7JWf6kzDKtKHQ5c9W5awDeQlj8C.3 2019-12-08T13:08:00.576 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.577+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:00.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.739+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:08:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.799+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:00.816 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.818+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:00.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.831+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:00.899 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.900+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:08:00.950 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.952+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:00.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:00.968+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T13:08:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.225', time=2019-12-08T15:07:25.460078+0000, chain.objs.size()=5 2019-12-08T13:08:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.1 2019-12-08T13:08:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.1_1 2019-12-08T13:08:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.2 2019-12-08T13:08:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.2_1 2019-12-08T13:08:01.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.3 2019-12-08T13:08:01.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_Zab00lBSJdav0fSW6Oenhr1OMqGgHD1', time=2019-12-08T15:07:26.252892+0000, chain.objs.size()=3 2019-12-08T13:08:01.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.hKScxI9eCLFR-2ChLy7DRrB761Zsa3M_0 2019-12-08T13:08:01.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.hKScxI9eCLFR-2ChLy7DRrB761Zsa3M_1 2019-12-08T13:08:01.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.hKScxI9eCLFR-2ChLy7DRrB761Zsa3M_2 2019-12-08T13:08:01.296 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.297+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:01.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.432+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:08:01.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.487+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:01.502 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.503+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.235', time=2019-12-08T15:07:26.453108+0000, chain.objs.size()=5 2019-12-08T13:08:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.1 2019-12-08T13:08:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.1_1 2019-12-08T13:08:01.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.2 2019-12-08T13:08:01.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.2_1 2019-12-08T13:08:01.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.518+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.3 2019-12-08T13:08:01.625 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.627+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:01.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.712+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:08:01.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.769+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:01.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.784+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:01.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-08T13:08:01.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_mHnbE6zkNxakmsr6Xa8LAF7efj6ysxo', time=2019-12-08T15:07:26.697179+0000, chain.objs.size()=2 2019-12-08T13:08:01.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.KgxYS_uhOMoV5KjBWkNyAUP2IdDgejB_1 2019-12-08T13:08:01.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.KgxYS_uhOMoV5KjBWkNyAUP2IdDgejB_2 2019-12-08T13:08:01.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.245', time=2019-12-08T15:07:27.468123+0000, chain.objs.size()=3 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-STANDARD.2~cUeluRY4r_xW6o_PPNlg3eUS1b0PHGP.1 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.796+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-STANDARD.2~cUeluRY4r_xW6o_PPNlg3eUS1b0PHGP.2 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-STANDARD.2~cUeluRY4r_xW6o_PPNlg3eUS1b0PHGP.3 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.196', time=2019-12-08T15:07:28.486314+0000, chain.objs.size()=5 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-FROZEN.2~QGtubHAr7yokqPZpbe9oNdJ-CL5fhXm.1 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.8__shadow_foo-FROZEN.2~QGtubHAr7yokqPZpbe9oNdJ-CL5fhXm.1_1 2019-12-08T13:08:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-FROZEN.2~QGtubHAr7yokqPZpbe9oNdJ-CL5fhXm.2 2019-12-08T13:08:01.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.8__shadow_foo-FROZEN.2~QGtubHAr7yokqPZpbe9oNdJ-CL5fhXm.2_1 2019-12-08T13:08:01.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:01.797+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.8__multipart_foo-FROZEN.2~QGtubHAr7yokqPZpbe9oNdJ-CL5fhXm.3 2019-12-08T13:08:02.169 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.170+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:02.275 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.276+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:08:02.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.313+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:02.326 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.327+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:02.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.339+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:02.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.339+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.85', time=2019-12-08T15:07:30.167136+0000, chain.objs.size()=3 2019-12-08T13:08:02.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.339+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.6__shadow_.I-rXwLWGiQvhurTvsDwxd2HudzTzM3L_0 2019-12-08T13:08:02.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.339+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.6__shadow_.I-rXwLWGiQvhurTvsDwxd2HudzTzM3L_1 2019-12-08T13:08:02.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.339+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.6__shadow_.I-rXwLWGiQvhurTvsDwxd2HudzTzM3L_2 2019-12-08T13:08:02.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.543+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:02.631 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.633+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:08:02.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.720+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:02.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.747+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:02.761 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.762+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:02.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.811+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:08:02.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.916+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:02.950 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.951+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:02.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.975+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:02.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.975+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_yPSUyCpQC60FN3PBLyUQVx6v_u8J7WT', time=2019-12-08T15:07:22.737381+0000, chain.objs.size()=2 2019-12-08T13:08:02.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.975+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.p5t1k8mILMhxGvLsu8mQwJkgh6AYJ6I_0 2019-12-08T13:08:02.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:02.975+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.p5t1k8mILMhxGvLsu8mQwJkgh6AYJ6I_1 2019-12-08T13:08:03.135 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.136+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:03.275 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.276+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:08:03.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.339+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:03.365 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.366+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-08T13:08:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_xckKzPdzBjQrBA8nLr74exDmUsr-3--', time=2019-12-08T15:07:22.158429+0000, chain.objs.size()=2 2019-12-08T13:08:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.eHt0bErFiblAIztAN9X1aOAa-qhUv93_0 2019-12-08T13:08:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.eHt0bErFiblAIztAN9X1aOAa-qhUv93_1 2019-12-08T13:08:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_i1Qy18Cc6lBeVIwvgv6tyFgk-KruHgx', time=2019-12-08T15:07:22.360877+0000, chain.objs.size()=1 2019-12-08T13:08:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.lAfhTfRoj_Na6QxX8VG52jbzjXjpLPD_1 2019-12-08T13:08:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.233', time=2019-12-08T15:07:26.093039+0000, chain.objs.size()=5 2019-12-08T13:08:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.1 2019-12-08T13:08:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.1_1 2019-12-08T13:08:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.2 2019-12-08T13:08:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.2_1 2019-12-08T13:08:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.386+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-LUKEWARM.2~rUyrXvUocO0dLST16nwTMP1qw-7h_JD.3 2019-12-08T13:08:03.671 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.673+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:03.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.798+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:08:03.843 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.845+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:03.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.861+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:03.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.874+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:03.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.874+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.71', time=2019-12-08T15:07:19.576508+0000, chain.objs.size()=4 2019-12-08T13:08:03.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.874+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.5__multipart_foo.2~QRVSKKVhzDfeaXVc-bMUacJELmZnbga.1 2019-12-08T13:08:03.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.874+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.5__multipart_foo.2~QRVSKKVhzDfeaXVc-bMUacJELmZnbga.2 2019-12-08T13:08:03.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.874+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.5__multipart_foo.2~QRVSKKVhzDfeaXVc-bMUacJELmZnbga.3 2019-12-08T13:08:03.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:03.874+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.5__multipart_foo.2~QRVSKKVhzDfeaXVc-bMUacJELmZnbga.4 2019-12-08T13:08:04.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.052+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:04.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.157+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:08:04.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.204+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:04.218 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.220+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:04.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.231+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:04.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.231+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.20', time=2019-12-08T15:07:14.962925+0000, chain.objs.size()=4 2019-12-08T13:08:04.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.231+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.2__multipart_foo.2~5r3QHSvtkezR_qYKqNEN-h-5UFur2pR.1 2019-12-08T13:08:04.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.231+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.2__multipart_foo.2~5r3QHSvtkezR_qYKqNEN-h-5UFur2pR.2 2019-12-08T13:08:04.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.231+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.2__multipart_foo.2~5r3QHSvtkezR_qYKqNEN-h-5UFur2pR.3 2019-12-08T13:08:04.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.231+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.2__multipart_foo.2~5r3QHSvtkezR_qYKqNEN-h-5UFur2pR.4 2019-12-08T13:08:04.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.435+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:04.535 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.536+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:08:04.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.596+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.613+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.622+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:04.670 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.671+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:08:04.739 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.740+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:04.760 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.762+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:04.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.779+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:04.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.779+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.108', time=2019-12-08T15:07:21.445244+0000, chain.objs.size()=2 2019-12-08T13:08:04.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.779+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.pNqwMP_b10v9jR5Pe91phu_0XAO4YQ5_0 2019-12-08T13:08:04.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.779+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.pNqwMP_b10v9jR5Pe91phu_0XAO4YQ5_1 2019-12-08T13:08:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:04.962+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:05.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.046+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:08:05.101 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.102+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.120 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.122+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:05.133 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.134+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T13:08:05.133 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.134+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_NUoLqkMZegUgjjhUZQ7a13zIigpQf_2', time=2019-12-08T15:07:23.572372+0000, chain.objs.size()=1 2019-12-08T13:08:05.133 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.134+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.7__shadow_.MssJU-q4HC2X0uiknhtQrQwylCV-3Gp_0 2019-12-08T13:08:05.134 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.135+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='_txstLzFKKBUulRwTm8q4kuuomQy1Vj_', time=2019-12-08T15:07:27.145232+0000, chain.objs.size()=1 2019-12-08T13:08:05.134 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.135+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_.nOhJHWAppaTLc-Wy3JJasFIF_NzTWvY_0 2019-12-08T13:08:05.195 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.196+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:05.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.285+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:08:05.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.330+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.347 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.348+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:05.357 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.358+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.417+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:08:05.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.459+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.474+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:05.481 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.483+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.508 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.510+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:08:05.573 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.575+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.590 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.592+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:05.601 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.602+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.795 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.796+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:08:05.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.849+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:05.885 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.886+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:05.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.904+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:05.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.904+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.40', time=2019-12-08T15:07:16.838567+0000, chain.objs.size()=4 2019-12-08T13:08:05.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.904+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.3__multipart_foo.2~7CS-tP2VjaxQm1EIlMQ6XLEX3dSWZJK.1 2019-12-08T13:08:05.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.904+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.3__multipart_foo.2~7CS-tP2VjaxQm1EIlMQ6XLEX3dSWZJK.2 2019-12-08T13:08:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.904+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.3__multipart_foo.2~7CS-tP2VjaxQm1EIlMQ6XLEX3dSWZJK.3 2019-12-08T13:08:05.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:05.904+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.3__multipart_foo.2~7CS-tP2VjaxQm1EIlMQ6XLEX3dSWZJK.4 2019-12-08T13:08:06.002 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.004+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:06.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.085+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:08:06.121 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.122+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:06.136 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.138+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:06.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:08:06.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.224', time=2019-12-08T15:07:25.294693+0000, chain.objs.size()=5 2019-12-08T13:08:06.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.1 2019-12-08T13:08:06.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.1_1 2019-12-08T13:08:06.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.2 2019-12-08T13:08:06.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__shadow_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.2_1 2019-12-08T13:08:06.156 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.155+0000 7f015810fec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:9161132f-da48-40b4-a853-f341a1971d48.4507.9__multipart_foo-FROZEN.2~nXP7bfbRWVTgJOMrkVsLuhgZmVGW-Cl.3 2019-12-08T13:08:06.387 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.389+0000 7f015810fec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:08:06.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.493+0000 7f015810fec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:08:06.574 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.575+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:06.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.598+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:06.610 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.611+0000 7f015810fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:06.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.704+0000 7f015810fec0 20 remove_watcher() i=0 2019-12-08T13:08:06.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.704+0000 7f015810fec0 2 removed watcher, disabling cache 2019-12-08T13:08:06.762 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.764+0000 7f015810fec0 20 remove_watcher() i=1 2019-12-08T13:08:06.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.807+0000 7f015810fec0 20 remove_watcher() i=2 2019-12-08T13:08:06.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.876+0000 7f015810fec0 20 remove_watcher() i=3 2019-12-08T13:08:06.938 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.939+0000 7f015810fec0 20 remove_watcher() i=4 2019-12-08T13:08:06.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:06.986+0000 7f015810fec0 20 remove_watcher() i=5 2019-12-08T13:08:07.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.047+0000 7f015810fec0 20 remove_watcher() i=6 2019-12-08T13:08:07.117 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.118+0000 7f015810fec0 20 remove_watcher() i=7 2019-12-08T13:08:07.125 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:08:07.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.232+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.251 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.252+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.251 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.252+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.261+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.260 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.261+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.265 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.267+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.266 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.267+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.274+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:08:07.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.454+0000 7f582a25fec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:08:07.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.454+0000 7f582a25fec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:08:07.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.454+0000 7f582a25fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:08:07.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.454+0000 7f582a25fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:08:07.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.454+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.468 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.469+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:08:07.468 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.469+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.477 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.478+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:08:07.477 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.478+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.492+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:08:07.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.492+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.511+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:08:07.510 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.511+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.519+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.519+0000 7f582a25fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:08:07.518 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.519+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.524 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.525+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.524 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.525+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.535+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:08:07.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.535+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.551+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:08:07.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.551+0000 7f582a25fec0 20 zonegroup default 2019-12-08T13:08:07.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.551+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.555 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.557+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.555 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.557+0000 7f582a25fec0 10 Cannot find current period zone using local zone 2019-12-08T13:08:07.555 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.557+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.564+0000 7f582a25fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:08:07.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.564+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.573 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.575+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:08:07.574 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.575+0000 7f582a25fec0 20 rados->read ofs=0 len=0 2019-12-08T13:08:07.582 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.584+0000 7f582a25fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:08:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.584+0000 7f582a25fec0 20 zone default found 2019-12-08T13:08:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.584+0000 7f582a25fec0 20 started sync module instance, tier type = 2019-12-08T13:08:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.584+0000 7f582a25fec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:08:07.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=0 2019-12-08T13:08:07.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=1 2019-12-08T13:08:07.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=2 2019-12-08T13:08:07.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=3 2019-12-08T13:08:07.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=4 2019-12-08T13:08:07.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=5 2019-12-08T13:08:07.926 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.926+0000 7f582a25fec0 20 add_watcher() i=6 2019-12-08T13:08:07.956 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.957+0000 7f582a25fec0 20 add_watcher() i=7 2019-12-08T13:08:07.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.957+0000 7f582a25fec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:08:07.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.958+0000 7f57e7fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:08:07.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:07.962+0000 7f5817c8a700 0 -- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55ee1096d310 msgr2=0x55ee109636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-08T13:08:08.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:08.006+0000 7f5817c8a700 0 -- 172.21.15.129:0/3042575386 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7f5758022470 msgr2=0x7f5758024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:08:09.596 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.596+0000 7f57567fc700 20 reqs_thread_entry: start 2019-12-08T13:08:09.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.596+0000 7f574ffff700 20 reqs_thread_entry: start 2019-12-08T13:08:09.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.596+0000 7f582a25fec0 20 init_complete bucket index max shards: 0 2019-12-08T13:08:09.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.596+0000 7f574e7fc700 20 reqs_thread_entry: start 2019-12-08T13:08:09.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.596+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:08:09.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.645+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:09.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.662+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:09.672 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.673+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:09.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.706+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:08:09.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.786+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:09.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.819+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:09.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.855+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:09.927 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.927+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:08:09.992 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:09.992+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.009 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.009+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.019 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.020+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.059 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.059+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:08:10.102 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.102+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.118 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.118+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.126 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.127+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.165+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:08:10.247 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.248+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.278+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.300+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.369 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.370+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:08:10.445 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.445+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.468 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.468+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.480 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.480+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.544+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:08:10.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.643+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.668 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.668+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.683+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.744 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.744+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:08:10.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.798+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.820+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.830+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.868+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:08:10.931 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.931+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.951 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.951+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:10.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.961+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:10.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:10.998+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:08:11.071 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.071+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.087 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.088+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.099+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.150 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.150+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:08:11.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.202+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.224 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.224+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.232+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.274 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.275+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:08:11.327 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.327+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.343+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.352 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.352+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.397 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.397+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:08:11.465 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.465+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.496 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.497+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.509+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.559 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.559+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:08:11.620 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.621+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.639+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.646 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.647+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.683+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:08:11.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.770+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.799 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.800+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.817+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.865+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:08:11.940 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.940+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:11.963 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.963+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:11.975+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.023 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.024+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:08:12.090 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.090+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.114 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.115+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.128+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.186+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:08:12.232 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.232+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.269 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.269+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.278+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.311+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:08:12.366 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.366+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.381 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.381+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.391 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.392+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.423 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.424+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:08:12.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.461+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.475+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.483 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.484+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.515 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.515+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:08:12.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.593+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.624 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.624+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.640 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.640+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.694 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.695+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:08:12.750 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.750+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.768 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.769+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.779 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.779+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.824 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.825+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:08:12.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.877+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.901+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:12.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.915+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:12.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:12.966+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:08:13.031 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.032+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.064 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.065+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.079 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.079+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.127+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:08:13.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.172+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.193+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.209 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.211+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.246+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:08:13.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.288+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.302 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.303+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.314+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.354+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:08:13.393 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.395+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.412+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.419 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.420+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.462 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.463+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:08:13.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.510+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.530 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.532+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.542+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.582 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.583+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:08:13.630 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.631+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.648+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.658 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.659+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.698+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:08:13.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.735+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.748 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.750+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.756 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.758+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.787+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:08:13.837 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.839+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.856 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.857+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:13.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.866+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.901+0000 7f582a25fec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:08:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.970+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:13.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:13.999+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:08:14.012 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.014+0000 7f582a25fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:08:14.106 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.108+0000 7f582a25fec0 20 remove_watcher() i=0 2019-12-08T13:08:14.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.108+0000 7f582a25fec0 2 removed watcher, disabling cache 2019-12-08T13:08:14.150 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.151+0000 7f582a25fec0 20 remove_watcher() i=1 2019-12-08T13:08:14.190 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.191+0000 7f582a25fec0 20 remove_watcher() i=2 2019-12-08T13:08:14.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.259+0000 7f582a25fec0 20 remove_watcher() i=3 2019-12-08T13:08:14.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.325+0000 7f582a25fec0 20 remove_watcher() i=4 2019-12-08T13:08:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.378+0000 7f582a25fec0 20 remove_watcher() i=5 2019-12-08T13:08:14.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.426+0000 7f582a25fec0 20 remove_watcher() i=6 2019-12-08T13:08:14.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:08:14.493+0000 7f582a25fec0 20 remove_watcher() i=7 2019-12-08T13:08:14.507 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-12-08T13:08:15.075 INFO:teuthology.orchestra.run.smithi129.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-12-08T13:08:18.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-12-08T13:08:18.274 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-12-08T13:08:19.376 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :clean UP-TO-DATE 2019-12-08T13:08:19.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-12-08T13:08:19.474 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-12-08T13:08:19.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-12-08T13:08:19.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-12-08T13:08:19.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-12-08T13:08:19.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-12-08T13:08:19.677 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-12-08T13:08:19.677 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-12-08T13:08:19.677 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-12-08T13:08:19.677 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-12-08T13:08:19.677 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-12-08T13:08:19.679 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-12-08T13:08:19.679 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-12-08T13:08:19.679 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-12-08T13:08:19.679 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-12-08T13:08:19.679 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-12-08T13:08:19.679 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-12-08T13:08:19.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-12-08T13:08:19.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-12-08T13:08:19.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-12-08T13:08:19.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-12-08T13:08:19.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-12-08T13:08:19.776 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-12-08T13:08:19.776 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-12-08T13:08:19.876 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-12-08T13:08:19.876 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-12-08T13:08:19.876 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-12-08T13:08:19.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-12-08T13:08:19.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-12-08T13:08:19.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-12-08T13:08:19.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-12-08T13:08:19.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-12-08T13:08:19.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-12-08T13:08:19.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-12-08T13:08:19.882 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-12-08T13:08:19.978 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-12-08T13:08:19.978 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-12-08T13:08:19.978 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-12-08T13:08:19.978 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-12-08T13:08:19.979 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-12-08T13:08:19.979 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-12-08T13:08:19.979 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-12-08T13:08:19.979 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-12-08T13:08:19.979 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-12-08T13:08:19.980 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-12-08T13:08:19.980 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-12-08T13:08:19.980 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-12-08T13:08:19.980 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-12-08T13:08:19.980 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-12-08T13:08:19.980 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-12-08T13:08:19.981 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-12-08T13:08:19.981 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-12-08T13:08:19.981 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-12-08T13:08:19.981 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-12-08T13:08:19.981 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-12-08T13:08:19.982 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-12-08T13:08:19.982 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-12-08T13:08:20.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-12-08T13:08:20.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-12-08T13:08:20.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-12-08T13:08:20.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-12-08T13:08:20.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-12-08T13:08:20.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-12-08T13:08:20.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-12-08T13:08:20.081 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-12-08T13:08:20.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-12-08T13:08:20.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-12-08T13:08:20.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-12-08T13:08:20.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-12-08T13:08:20.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-12-08T13:08:20.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-12-08T13:08:20.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-12-08T13:08:20.180 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-12-08T13:08:20.180 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-12-08T13:08:20.180 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-12-08T13:08:20.180 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-12-08T13:08:20.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-12-08T13:08:20.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-12-08T13:08:20.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-12-08T13:08:20.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-12-08T13:08:20.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-12-08T13:08:20.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-12-08T13:08:20.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-12-08T13:08:20.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-12-08T13:08:20.279 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-12-08T13:08:20.280 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-12-08T13:08:20.281 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-12-08T13:08:20.282 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-12-08T13:08:20.283 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-12-08T13:08:20.284 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-12-08T13:08:20.284 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-12-08T13:08:20.284 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-12-08T13:08:20.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-12-08T13:08:20.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-12-08T13:08:20.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-12-08T13:08:20.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-12-08T13:08:20.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-12-08T13:08:20.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-12-08T13:08:20.379 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-12-08T13:08:20.380 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-12-08T13:08:20.380 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-12-08T13:08:20.380 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-12-08T13:08:20.380 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-12-08T13:08:20.380 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-12-08T13:08:20.380 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-12-08T13:08:20.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-12-08T13:08:20.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-12-08T13:08:20.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-12-08T13:08:20.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-12-08T13:08:20.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-12-08T13:08:20.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-12-08T13:08:20.477 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-12-08T13:08:20.477 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-12-08T13:08:20.575 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-12-08T13:08:20.575 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-12-08T13:08:20.575 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-12-08T13:08:20.575 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-12-08T13:08:20.575 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-12-08T13:08:20.576 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-12-08T13:08:20.576 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-12-08T13:08:20.576 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-12-08T13:08:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-12-08T13:08:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-12-08T13:08:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-12-08T13:08:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-12-08T13:08:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-12-08T13:08:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-12-08T13:08:20.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-12-08T13:08:20.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-12-08T13:08:20.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-12-08T13:08:20.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-12-08T13:08:20.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-12-08T13:08:20.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-12-08T13:08:20.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-12-08T13:08:20.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-12-08T13:08:20.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-12-08T13:08:20.776 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-12-08T13:08:20.776 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-12-08T13:08:20.776 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-12-08T13:08:20.776 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-12-08T13:08:20.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-12-08T13:08:20.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-12-08T13:08:20.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-12-08T13:08:20.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-12-08T13:08:20.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-12-08T13:08:20.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-12-08T13:08:20.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-12-08T13:08:20.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-12-08T13:08:20.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-12-08T13:08:20.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-12-08T13:08:20.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-12-08T13:08:20.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-12-08T13:08:20.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-12-08T13:08:20.977 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-12-08T13:08:21.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-12-08T13:08:21.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-12-08T13:08:21.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-12-08T13:08:21.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-12-08T13:08:21.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-12-08T13:08:21.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-12-08T13:08:21.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-12-08T13:08:21.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-12-08T13:08:21.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-12-08T13:08:21.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-12-08T13:08:21.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-12-08T13:08:21.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-12-08T13:08:21.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-12-08T13:08:21.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-12-08T13:08:21.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-12-08T13:08:21.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-12-08T13:08:21.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-12-08T13:08:21.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-12-08T13:08:21.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-12-08T13:08:21.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-12-08T13:08:21.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-12-08T13:08:21.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-12-08T13:08:21.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-12-08T13:08:21.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-12-08T13:08:21.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-12-08T13:08:21.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-12-08T13:08:21.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-12-08T13:08:22.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-12-08T13:08:22.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-12-08T13:08:22.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-12-08T13:08:22.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-12-08T13:08:22.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-12-08T13:08:22.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-12-08T13:08:22.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-12-08T13:08:22.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-12-08T13:08:22.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-12-08T13:08:22.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-12-08T13:08:22.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-12-08T13:08:22.178 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-12-08T13:08:22.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-12-08T13:08:22.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-12-08T13:08:22.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-12-08T13:08:22.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-12-08T13:08:22.179 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-12-08T13:08:22.180 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-12-08T13:08:22.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-12-08T13:08:22.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-12-08T13:08:22.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-12-08T13:08:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-12-08T13:08:22.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-12-08T13:08:22.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-12-08T13:08:22.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-12-08T13:08:22.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-12-08T13:08:22.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-12-08T13:08:22.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-12-08T13:08:22.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-12-08T13:08:22.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-12-08T13:08:22.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-12-08T13:08:22.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-12-08T13:08:22.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-12-08T13:08:22.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-12-08T13:08:22.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-12-08T13:08:22.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-12-08T13:08:22.475 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-12-08T13:08:22.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-12-08T13:08:22.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-12-08T13:08:22.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-12-08T13:08:22.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-12-08T13:08:22.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-12-08T13:08:22.476 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-12-08T13:08:22.576 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-12-08T13:08:22.576 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-12-08T13:08:22.576 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-12-08T13:08:22.577 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-12-08T13:08:22.578 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-12-08T13:08:22.578 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-12-08T13:08:22.578 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-12-08T13:08:22.578 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-12-08T13:08:22.578 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-12-08T13:08:22.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-12-08T13:08:22.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-12-08T13:08:22.675 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-12-08T13:08:22.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-12-08T13:08:22.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-12-08T13:08:22.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-12-08T13:08:22.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-12-08T13:08:22.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-12-08T13:08:22.676 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-12-08T13:08:22.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-12-08T13:08:22.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-12-08T13:08:22.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-12-08T13:08:22.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-12-08T13:08:22.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-12-08T13:08:22.775 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-12-08T13:08:22.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-12-08T13:08:22.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-12-08T13:08:22.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-12-08T13:08:22.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-12-08T13:08:22.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-12-08T13:08:22.875 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-12-08T13:08:22.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-12-08T13:08:22.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-12-08T13:08:22.877 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-12-08T13:08:22.878 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-12-08T13:08:22.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-12-08T13:08:22.879 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-12-08T13:08:22.975 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-12-08T13:08:22.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-12-08T13:08:22.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-12-08T13:08:22.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-12-08T13:08:22.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-12-08T13:08:22.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-12-08T13:08:22.976 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-12-08T13:08:22.977 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-12-08T13:08:22.977 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-12-08T13:08:23.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-12-08T13:08:23.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-12-08T13:08:23.075 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-12-08T13:08:23.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-12-08T13:08:23.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-12-08T13:08:23.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-12-08T13:08:23.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-12-08T13:08:23.076 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-12-08T13:08:23.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-12-08T13:08:23.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-12-08T13:08:23.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-12-08T13:08:23.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-12-08T13:08:23.077 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-12-08T13:08:23.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-12-08T13:08:23.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-12-08T13:08:23.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-12-08T13:08:23.078 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-12-08T13:08:23.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-12-08T13:08:23.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-12-08T13:08:23.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-12-08T13:08:23.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-12-08T13:08:23.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-12-08T13:08:23.079 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-12-08T13:08:23.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-12-08T13:08:23.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-12-08T13:08:23.080 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-12-08T13:08:23.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-12-08T13:08:23.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-12-08T13:08:23.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-12-08T13:08:23.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-12-08T13:08:23.175 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-12-08T13:08:23.176 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-12-08T13:08:23.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-12-08T13:08:23.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-12-08T13:08:23.177 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-12-08T13:08:23.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-12-08T13:08:23.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-12-08T13:08:23.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-12-08T13:08:23.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-12-08T13:08:23.275 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-12-08T13:08:23.276 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-12-08T13:08:23.277 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-12-08T13:08:23.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-12-08T13:08:23.278 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-12-08T13:08:23.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-12-08T13:08:23.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-12-08T13:08:23.375 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-12-08T13:08:23.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-12-08T13:08:23.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-12-08T13:08:23.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-12-08T13:08:23.376 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-12-08T13:08:23.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-12-08T13:08:23.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-12-08T13:08:23.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-12-08T13:08:23.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-12-08T13:08:23.377 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-12-08T13:08:23.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-12-08T13:08:23.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-12-08T13:08:23.378 INFO:teuthology.orchestra.run.smithi129.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-12-08T13:08:31.975 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :compileJava 2019-12-08T13:08:31.975 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :processResources 2019-12-08T13:08:31.975 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :classes 2019-12-08T13:08:32.475 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:08:32.476 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :compileTestJava 2019-12-08T13:08:32.476 INFO:teuthology.orchestra.run.smithi129.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-08T13:08:32.476 INFO:teuthology.orchestra.run.smithi129.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-08T13:08:32.476 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:08:32.476 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :processTestResources NO-SOURCE 2019-12-08T13:08:32.476 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :testClasses 2019-12-08T13:08:40.274 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:08:40.275 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :test 2019-12-08T13:08:40.275 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:08:40.275 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-12-08T13:09:22.938 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:25:54.369 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:22.942 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:25:54.373 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.012 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:25:54.441 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.014 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:25:54.589 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.014 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:25:54.443 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.020 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:25:54.595 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.080 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:25:54.511 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.150 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:25:54.725 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:23.154 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:25:54.583 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:09:36.675 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:09:36.675 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-12-08T13:09:43.175 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:09:43.175 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-12-08T13:09:46.075 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:09:46.075 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-12-08T13:09:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:09:52.175 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-12-08T13:11:03.776 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:11:03.776 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-12-08T13:11:11.276 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:11:11.276 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-12-08T13:11:11.476 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:11:11.476 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-12-08T13:11:12.276 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:11:12.276 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-12-08T13:11:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:11:16.176 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-12-08T13:11:22.776 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:11:22.776 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-12-08T13:12:09.376 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:12:09.376 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-12-08T13:12:27.276 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:12:27.276 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-12-08T13:15:39.016 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:32:10.444 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.024 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:32:10.452 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.032 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:32:10.459 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.038 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:32:10.624 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.046 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:32:10.466 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.054 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:32:10.640 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.272 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:32:10.698 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.291 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:32:10.876 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:15:39.301 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:32:10.729 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:16:50.978 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:16:50.978 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-12-08T13:17:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:17:04.978 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-12-08T13:17:06.878 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:17:06.878 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-12-08T13:17:12.378 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:17:12.378 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-12-08T13:17:16.378 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:17:16.378 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-12-08T13:17:28.378 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:17:28.378 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-12-08T13:18:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:18:05.478 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-12-08T13:18:13.578 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:18:13.578 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-12-08T13:18:13.978 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:18:13.978 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-12-08T13:18:15.278 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:18:15.278 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-12-08T13:19:00.293 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T13:19:00.229+0000 e17ba700 -1 handle_data failed with exit code -104 2019-12-08T13:19:01.878 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:19:01.878 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-12-08T13:19:09.378 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:19:09.378 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-12-08T13:20:09.578 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:20:09.579 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-12-08T13:20:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:20:20.879 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-12-08T13:20:50.046 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:37:21.473 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.051 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:37:21.478 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.060 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:37:21.485 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.064 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:37:21.489 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.070 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:37:21.664 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.078 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:37:21.672 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.215 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:37:21.642 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.221 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:37:21.646 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:50.308 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:37:21.902 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:20:55.678 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:20:55.679 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-12-08T13:21:03.678 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:21:03.679 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-12-08T13:26:03.702 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:42:35.129 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.713 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:42:35.139 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.713 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:42:35.138 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.716 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:42:35.302 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.718 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:42:35.143 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.733 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:42:35.319 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.893 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:42:35.318 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.904 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:42:35.331 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:03.973 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:42:35.559 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:05.956 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:42:37.382 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:05.998 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:42:37.584 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:05.998 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:42:37.425 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:06.002 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:42:37.588 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:06.015 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:42:37.440 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:06.016 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:42:37.443 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:06.021 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:42:37.606 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:06.031 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:42:37.456 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:26:06.045 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:42:37.470 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:14.901 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:47:46.327 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:14.905 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:47:46.332 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:14.915 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:47:46.340 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:14.919 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:47:46.344 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:14.924 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:47:46.503 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:14.934 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:47:46.512 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:15.086 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:47:46.513 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:15.100 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:47:46.525 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:31:15.190 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:47:46.767 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:17.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:34:17.579 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-12-08T13:34:25.879 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:34:25.879 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-12-08T13:34:27.340 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:50:58.766 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.376 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:50:58.803 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.380 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:50:58.805 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.395 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:50:58.819 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.435 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:50:59.011 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.442 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:50:59.017 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.461 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:50:58.887 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.510 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:50:58.935 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:34:27.541 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:50:59.117 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:12.679 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:36:12.679 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-12-08T13:36:20.779 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:36:20.779 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-12-08T13:36:26.063 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:52:57.490 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.074 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:52:57.500 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.083 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:52:57.508 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.089 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:52:57.667 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.090 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:52:57.514 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.100 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:52:57.678 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.155 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:52:57.581 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.199 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:52:57.623 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:36:26.202 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:52:57.779 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:37:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:37:11.180 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-12-08T13:37:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:37:15.579 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-12-08T13:37:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:37:25.679 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-12-08T13:37:42.379 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:37:42.379 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-12-08T13:37:45.979 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:37:45.979 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-12-08T13:38:05.179 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:38:05.180 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-12-08T13:38:10.679 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:38:10.679 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-12-08T13:38:29.479 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:38:29.479 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-12-08T13:38:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:38:33.780 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-12-08T13:39:09.479 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:39:09.479 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-12-08T13:39:17.879 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:39:17.879 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-12-08T13:39:53.679 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:39:53.679 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-12-08T13:39:59.779 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:39:59.779 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-12-08T13:41:03.979 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:41:03.979 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-12-08T13:41:18.379 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:41:18.379 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-12-08T13:41:38.522 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:58:09.949 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.529 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:58:09.955 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.538 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:58:09.963 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.543 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:58:09.967 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.552 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:58:10.130 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.572 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:58:10.149 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.736 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:58:10.163 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.752 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:58:10.177 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:41:38.838 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:58:10.416 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:42:12.579 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-12-08T13:42:33.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:42:33.579 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-12-08T13:42:48.650 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:59:20.077 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.685 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:59:20.112 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.695 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:59:20.120 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.712 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:59:20.137 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.739 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:59:20.318 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.740 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:00:59:20.167 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.744 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:59:20.324 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.770 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:00:59:20.194 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:42:48.796 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:00:59:20.376 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:43:43.578 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:43:43.579 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-12-08T13:46:50.457 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:03:21.884 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.459 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:03:21.886 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.468 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:03:21.894 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.472 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:03:21.897 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.484 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:03:22.069 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.497 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:03:22.082 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.615 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:03:22.040 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.626 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:03:22.053 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:46:50.728 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:03:22.313 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:09.993 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:07:41.419 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.015 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:07:41.442 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.037 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:07:41.462 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.044 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:07:41.629 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.055 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:07:41.481 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.063 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:07:41.645 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.163 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:07:41.589 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.165 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:07:41.750 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:10.184 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:07:41.609 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:51:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:51:13.779 INFO:teuthology.orchestra.run.smithi129.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-12-08T13:52:02.475 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:08:33.902 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.483 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:08:33.910 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.491 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:08:33.916 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.496 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:08:33.921 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.508 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:08:34.092 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.518 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:08:34.102 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.537 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:08:33.964 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.545 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:08:33.970 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:02.583 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:08:34.167 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:52:29.879 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:52:29.879 INFO:teuthology.orchestra.run.smithi129.stdout:---------------------------------------------------------------------- 2019-12-08T13:52:29.879 INFO:teuthology.orchestra.run.smithi129.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-12-08T13:52:29.879 INFO:teuthology.orchestra.run.smithi129.stdout:---------------------------------------------------------------------- 2019-12-08T13:52:30.147 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:52:30.147 INFO:teuthology.orchestra.run.smithi129.stdout:BUILD SUCCESSFUL in 44m 15s 2019-12-08T13:52:30.147 INFO:teuthology.orchestra.run.smithi129.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-12-08T13:52:30.471 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:52:30.630 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.626+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.655+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.655+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.663+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.663+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.671 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.668+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.671 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.668+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.681 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.678+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:30.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.860+0000 7f1ad3379ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:52:30.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.860+0000 7f1ad3379ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:52:30.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.860+0000 7f1ad3379ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:52:30.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.860+0000 7f1ad3379ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:52:30.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.860+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.872+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:30.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.872+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.885+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:52:30.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.886+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.899 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.895+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:30.899 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.895+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.912+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:52:30.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.912+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.919+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.919+0000 7f1ad3379ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:52:30.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.919+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.931 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.928+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.932 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.928+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.937+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.937+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.950+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:52:30.954 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.950+0000 7f1ad3379ec0 20 zonegroup default 2019-12-08T13:52:30.954 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.950+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.954+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.954+0000 7f1ad3379ec0 10 Cannot find current period zone using local zone 2019-12-08T13:52:30.957 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.954+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.959+0000 7f1ad3379ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:30.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.959+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.966+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:30.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.966+0000 7f1ad3379ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.982+0000 7f1ad3379ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:52:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.982+0000 7f1ad3379ec0 20 zone default found 2019-12-08T13:52:30.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.982+0000 7f1ad3379ec0 20 started sync module instance, tier type = 2019-12-08T13:52:30.986 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:30.982+0000 7f1ad3379ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:52:31.328 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=0 2019-12-08T13:52:31.328 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=1 2019-12-08T13:52:31.328 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=2 2019-12-08T13:52:31.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=3 2019-12-08T13:52:31.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=4 2019-12-08T13:52:31.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=5 2019-12-08T13:52:31.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.325+0000 7f1ad3379ec0 20 add_watcher() i=6 2019-12-08T13:52:31.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.366+0000 7f1ad3379ec0 20 add_watcher() i=7 2019-12-08T13:52:31.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.366+0000 7f1ad3379ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:52:31.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.366+0000 7f1a90fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:52:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.374+0000 7f1abfda2700 0 -- 172.21.15.129:0/1772595867 >> v2:172.21.15.181:3300/0 conn(0x55d11e482490 msgr2=0x55d11e4848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T13:52:31.414 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:31.411+0000 7f1ac05a3700 0 -- 172.21.15.129:0/1396006210 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7f1a04022490 msgr2=0x7f1a04024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:52:32.694 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.691+0000 7f1a02ffd700 20 reqs_thread_entry: start 2019-12-08T13:52:32.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.691+0000 7f1a00ff9700 20 reqs_thread_entry: start 2019-12-08T13:52:32.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.691+0000 7f1ad3379ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:52:32.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.691+0000 7f19f6ffd700 20 reqs_thread_entry: start 2019-12-08T13:52:32.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.691+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:52:32.748 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.745+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:32.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.764+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:32.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:32.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.430', time=2019-12-08T15:17:00.293186+0000, chain.objs.size()=5 2019-12-08T13:52:32.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.19__shadow_.d6abxTWNuV0e2aBVpMHKCNr5EaVF-lI_1 2019-12-08T13:52:32.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.19__shadow_.d6abxTWNuV0e2aBVpMHKCNr5EaVF-lI_2 2019-12-08T13:52:32.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.19__shadow_.d6abxTWNuV0e2aBVpMHKCNr5EaVF-lI_3 2019-12-08T13:52:32.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.19__shadow_.d6abxTWNuV0e2aBVpMHKCNr5EaVF-lI_4 2019-12-08T13:52:32.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:32.779+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.19__shadow_.d6abxTWNuV0e2aBVpMHKCNr5EaVF-lI_5 2019-12-08T13:52:33.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.064+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.188+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:52:33.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.251+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:33.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.268+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.277+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:33.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.320+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:52:33.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.398+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:33.434 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.431+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:33.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:33.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.454', time=2019-12-08T15:15:54.561803+0000, chain.objs.size()=9 2019-12-08T13:52:33.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.1 2019-12-08T13:52:33.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.1_1 2019-12-08T13:52:33.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.2 2019-12-08T13:52:33.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.2_1 2019-12-08T13:52:33.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.3 2019-12-08T13:52:33.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.3_1 2019-12-08T13:52:33.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.4 2019-12-08T13:52:33.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.4_1 2019-12-08T13:52:33.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.459+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~9lNMXG1KRhHOlYJzbMNTQb64-YmJwdg.5 2019-12-08T13:52:33.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.675+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.811+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:52:33.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.856+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:33.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.874+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.884+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:33.933 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.930+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:52:33.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.968+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:33.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:33.995+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:34.023 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.417', time=2019-12-08T15:12:12.966928+0000, chain.objs.size()=9 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.1 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.1_1 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.2 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.2_1 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.3 2019-12-08T13:52:34.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.3_1 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.4 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.4_1 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.5 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.415', time=2019-12-08T15:12:21.072444+0000, chain.objs.size()=9 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.1 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.1_1 2019-12-08T13:52:34.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.019+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.2 2019-12-08T13:52:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.164+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.2_1 2019-12-08T13:52:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.236+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.3 2019-12-08T13:52:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.236+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.3_1 2019-12-08T13:52:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.236+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.4 2019-12-08T13:52:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.236+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__shadow_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.4_1 2019-12-08T13:52:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.236+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.17__multipart_key1.2~Oxs0vYv3XBO8pRFDBFiQ4UOTr3w7cop.5 2019-12-08T13:52:34.397 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.394+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.533+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:52:34.573 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.569+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.588+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:34.601 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.598+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.636+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:52:34.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.674+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:34.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.692+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:34.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.702+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:34.739 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.736+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:52:34.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.777+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.792+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:34.813 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:34.813 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb', time=2019-12-08T15:37:36.247178+0000, chain.objs.size()=22 2019-12-08T13:52:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.1 2019-12-08T13:52:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.2 2019-12-08T13:52:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.3 2019-12-08T13:52:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.4 2019-12-08T13:52:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.5 2019-12-08T13:52:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.6 2019-12-08T13:52:34.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.7 2019-12-08T13:52:34.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.8 2019-12-08T13:52:34.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.9 2019-12-08T13:52:34.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.10 2019-12-08T13:52:34.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.11 2019-12-08T13:52:34.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:34.809+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.5Jl3LnDXbkt5jq8MrBZh-FyU4b3wbmX.12 2019-12-08T13:52:35.009 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.006+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.M3JcjryoeG9nzxHq4lR56mgZQ6pGh9e.13 2019-12-08T13:52:35.010 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.006+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.MxBMTrWinZ_Hvi-XUofz3DjvFKTypWG.14 2019-12-08T13:52:35.010 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.006+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.StdisAGpcZ5HL_O9-ACYPvhKAYlcUWo.15 2019-12-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.043+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.YjvI4DPlGRG_A4Y7zQYcIVfVOEAE6gp.16 2019-12-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.043+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.X3eWnY0sYleTPF4737TFlyZFaTLunty.17 2019-12-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.043+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.sZ8_RQkJUiQZHcaM-xBIaQAeWG-vcgB.18 2019-12-08T13:52:35.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.151+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.p7ynjGgkCaN7bO9R7ND-fEOZxivTszB.19 2019-12-08T13:52:35.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.151+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.UkOkmQLS_62dOlVMg9OJ0AeWDf4lH2b.20 2019-12-08T13:52:35.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.151+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.21 2019-12-08T13:52:35.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.151+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.31__multipart_key1.2~45MY88sIX0X4rNW_WYXx-J6PEHof0vb.22 2019-12-08T13:52:35.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.340+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:35.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.431+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:52:35.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.472+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:35.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.487+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:35.502 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:35.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.460', time=2019-12-08T15:16:21.846164+0000, chain.objs.size()=7 2019-12-08T13:52:35.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.1 2019-12-08T13:52:35.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.1_1 2019-12-08T13:52:35.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.1_2 2019-12-08T13:52:35.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.2 2019-12-08T13:52:35.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.2_1 2019-12-08T13:52:35.504 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.2_2 2019-12-08T13:52:35.504 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.499+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~rr68yMxVwaCm9HfnFMtJ228I8GPx10c.3 2019-12-08T13:52:35.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.774+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:35.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.864+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:52:35.907 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.904+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:35.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.919+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:35.940 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T13:52:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.472', time=2019-12-08T15:16:53.467167+0000, chain.objs.size()=7 2019-12-08T13:52:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.1 2019-12-08T13:52:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.1_1 2019-12-08T13:52:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.1_2 2019-12-08T13:52:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.2 2019-12-08T13:52:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.2_1 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.2_2 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~6a8zmlkImEjKoRQeiVaenwgrZNyN8qA.3 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.628', time=2019-12-08T15:34:16.451490+0000, chain.objs.size()=16 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.1 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.1_1 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.1_2 2019-12-08T13:52:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.2 2019-12-08T13:52:35.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:35.937+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.2_1 2019-12-08T13:52:36.148 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.145+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.2_2 2019-12-08T13:52:36.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.3 2019-12-08T13:52:36.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.3_1 2019-12-08T13:52:36.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.3_2 2019-12-08T13:52:36.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.4 2019-12-08T13:52:36.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.4_1 2019-12-08T13:52:36.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.4_2 2019-12-08T13:52:36.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.5 2019-12-08T13:52:36.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.5_1 2019-12-08T13:52:36.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.5_2 2019-12-08T13:52:36.343 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.338+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~t9qmX5R_myhBN6H8BH_xToeL7PS1ewJ.6 2019-12-08T13:52:36.775 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.772+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:36.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.873+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:52:36.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.938+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:36.969 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.966+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:36.982 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:36.979+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:37.038 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.035+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:52:37.131 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.128+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:37.156 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.152+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:37.183 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T13:52:37.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.386', time=2019-12-08T15:11:06.128868+0000, chain.objs.size()=9 2019-12-08T13:52:37.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__multipart_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.1 2019-12-08T13:52:37.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__shadow_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.1_1 2019-12-08T13:52:37.184 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__multipart_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.2 2019-12-08T13:52:37.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__shadow_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.2_1 2019-12-08T13:52:37.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__multipart_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.3 2019-12-08T13:52:37.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__shadow_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.3_1 2019-12-08T13:52:37.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__multipart_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.4 2019-12-08T13:52:37.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__shadow_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.4_1 2019-12-08T13:52:37.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.15__multipart_key1.2~1zw8R8-nzErH95ZJG3vrOm5ZK1P6lzd.5 2019-12-08T13:52:37.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.651', time=2019-12-08T15:36:14.597367+0000, chain.objs.size()=9 2019-12-08T13:52:37.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__multipart_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.1 2019-12-08T13:52:37.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__shadow_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.1_1 2019-12-08T13:52:37.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.180+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__multipart_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.2 2019-12-08T13:52:37.292 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.289+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__shadow_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.2_1 2019-12-08T13:52:37.347 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.344+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__multipart_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.3 2019-12-08T13:52:37.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.539+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__shadow_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.3_1 2019-12-08T13:52:37.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.539+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__multipart_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.4 2019-12-08T13:52:37.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.539+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__shadow_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.4_1 2019-12-08T13:52:37.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.576+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.29__multipart_key1.2~VYFM8fwv6NeWrjJfCW-p7Xvk-3ep6bp.5 2019-12-08T13:52:37.776 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.773+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:37.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.886+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:52:37.931 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.928+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:37.952 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.949+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:37.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:37.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.553', time=2019-12-08T15:20:12.110151+0000, chain.objs.size()=9 2019-12-08T13:52:37.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__multipart_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.1 2019-12-08T13:52:37.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__shadow_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.1_1 2019-12-08T13:52:37.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__multipart_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.2 2019-12-08T13:52:37.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__shadow_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.2_1 2019-12-08T13:52:37.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__multipart_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.3 2019-12-08T13:52:37.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__shadow_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.3_1 2019-12-08T13:52:37.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__multipart_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.4 2019-12-08T13:52:37.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__shadow_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.4_1 2019-12-08T13:52:37.971 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:37.966+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.26__multipart_file.mpg.2~uetsOXJsGA3tsnfOFDPUG8RHszFpg8-.5 2019-12-08T13:52:38.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.227+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:38.322 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.319+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:52:38.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.361+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:38.381 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.378+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:38.398 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.612', time=2019-12-08T15:29:46.764922+0000, chain.objs.size()=19 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.1 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.1_1 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.2 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.2_1 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.3 2019-12-08T13:52:38.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.3_1 2019-12-08T13:52:38.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.4 2019-12-08T13:52:38.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.4_1 2019-12-08T13:52:38.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.5 2019-12-08T13:52:38.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.5_1 2019-12-08T13:52:38.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.6 2019-12-08T13:52:38.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.395+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.6_1 2019-12-08T13:52:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.605+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.7 2019-12-08T13:52:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.605+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.7_1 2019-12-08T13:52:38.620 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.615+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.8 2019-12-08T13:52:38.621 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.615+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.8_1 2019-12-08T13:52:38.621 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.615+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.9 2019-12-08T13:52:38.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.696+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.9_1 2019-12-08T13:52:38.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.696+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~oNzepS-B_86p5g88I9PL-3xaXJf-E7t.10 2019-12-08T13:52:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:38.954+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:39.075 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.072+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:52:39.109 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.106+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:39.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.124+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:39.136 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.133+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:39.167 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.164+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:52:39.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.242+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:39.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.274+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-08T13:52:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.528', time=2019-12-08T15:19:04.103335+0000, chain.objs.size()=9 2019-12-08T13:52:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__multipart_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.1 2019-12-08T13:52:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__shadow_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.1_1 2019-12-08T13:52:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__multipart_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.2 2019-12-08T13:52:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__shadow_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.2_1 2019-12-08T13:52:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__multipart_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.3 2019-12-08T13:52:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.319+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__shadow_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.3_1 2019-12-08T13:52:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__multipart_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.4 2019-12-08T13:52:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__shadow_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.4_1 2019-12-08T13:52:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.25__multipart_key1.2~VTFCYGvR5sxH5YoxL2m9u2OrVoaoN9K.5 2019-12-08T13:52:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.587', time=2019-12-08T15:25:55.899781+0000, chain.objs.size()=21 2019-12-08T13:52:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.1 2019-12-08T13:52:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.1_1 2019-12-08T13:52:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.320+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.2 2019-12-08T13:52:39.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.581+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.2_1 2019-12-08T13:52:39.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.3 2019-12-08T13:52:39.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.3_1 2019-12-08T13:52:39.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.4 2019-12-08T13:52:39.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.4_1 2019-12-08T13:52:39.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.5 2019-12-08T13:52:39.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.5_1 2019-12-08T13:52:39.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.6 2019-12-08T13:52:39.687 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.683+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.6_1 2019-12-08T13:52:39.732 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.729+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.7 2019-12-08T13:52:39.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.729+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.7_1 2019-12-08T13:52:39.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:39.729+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.8 2019-12-08T13:52:40.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.8_1 2019-12-08T13:52:40.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.9 2019-12-08T13:52:40.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.9_1 2019-12-08T13:52:40.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.10 2019-12-08T13:52:40.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.10_1 2019-12-08T13:52:40.035 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~jVPQ9D44bDQKmvnkoYmlI7tiXGkgemi.11 2019-12-08T13:52:40.036 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.636', time=2019-12-08T15:34:19.650670+0000, chain.objs.size()=16 2019-12-08T13:52:40.036 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.1 2019-12-08T13:52:40.036 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.031+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.1_1 2019-12-08T13:52:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.220+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.1_2 2019-12-08T13:52:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.220+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.2 2019-12-08T13:52:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.220+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.2_1 2019-12-08T13:52:40.334 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.2_2 2019-12-08T13:52:40.334 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.3 2019-12-08T13:52:40.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.3_1 2019-12-08T13:52:40.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.3_2 2019-12-08T13:52:40.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.4 2019-12-08T13:52:40.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.4_1 2019-12-08T13:52:40.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.331+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.4_2 2019-12-08T13:52:40.446 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.443+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.5 2019-12-08T13:52:40.447 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.443+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.5_1 2019-12-08T13:52:40.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.451+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.5_2 2019-12-08T13:52:40.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.451+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~1iQj4WA2kJ6097WAW92HZZOSVn9TLNs.6 2019-12-08T13:52:40.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.698+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:40.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.823+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:52:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.869+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:40.888 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.885+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:40.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:40.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~xUqJg6JSpJO6n7CmuxMZXMGZTDArg8Z', time=2019-12-08T15:20:59.758452+0000, chain.objs.size()=5 2019-12-08T13:52:40.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.27__multipart_key1.2~xUqJg6JSpJO6n7CmuxMZXMGZTDArg8Z.7 2019-12-08T13:52:40.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.27__shadow_key1.2~xUqJg6JSpJO6n7CmuxMZXMGZTDArg8Z.7_1 2019-12-08T13:52:40.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.27__multipart_key1.2~xUqJg6JSpJO6n7CmuxMZXMGZTDArg8Z.10 2019-12-08T13:52:40.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.27__shadow_key1.2~xUqJg6JSpJO6n7CmuxMZXMGZTDArg8Z.10_1 2019-12-08T13:52:40.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:40.900+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.27__multipart_key1.2~xUqJg6JSpJO6n7CmuxMZXMGZTDArg8Z.13 2019-12-08T13:52:41.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.123+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:41.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.227+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:52:41.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.273+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:41.294 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.291+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:41.319 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:41.319 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.600', time=2019-12-08T15:27:54.315468+0000, chain.objs.size()=21 2019-12-08T13:52:41.319 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.1 2019-12-08T13:52:41.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.1_1 2019-12-08T13:52:41.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.2 2019-12-08T13:52:41.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.2_1 2019-12-08T13:52:41.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.3 2019-12-08T13:52:41.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.3_1 2019-12-08T13:52:41.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.4 2019-12-08T13:52:41.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.4_1 2019-12-08T13:52:41.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.5 2019-12-08T13:52:41.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.5_1 2019-12-08T13:52:41.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.6 2019-12-08T13:52:41.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.315+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.6_1 2019-12-08T13:52:41.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.7 2019-12-08T13:52:41.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.7_1 2019-12-08T13:52:41.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.8 2019-12-08T13:52:41.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.8_1 2019-12-08T13:52:41.545 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.9 2019-12-08T13:52:41.545 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.9_1 2019-12-08T13:52:41.545 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.10 2019-12-08T13:52:41.545 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.540+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.10_1 2019-12-08T13:52:41.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.657+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~pOv6ims4_WYg6lgWuGqIIudycGzuzn6.11 2019-12-08T13:52:41.880 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.877+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:41.981 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:41.978+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:52:42.023 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.020+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:42.037 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.034+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:42.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.043+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:42.085 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.082+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:52:42.132 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.129+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:42.155 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.152+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:42.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:42.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.620', time=2019-12-08T15:31:46.588899+0000, chain.objs.size()=16 2019-12-08T13:52:42.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.1 2019-12-08T13:52:42.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.1_1 2019-12-08T13:52:42.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.1_2 2019-12-08T13:52:42.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.2 2019-12-08T13:52:42.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.2_1 2019-12-08T13:52:42.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.2_2 2019-12-08T13:52:42.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.3 2019-12-08T13:52:42.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.3_1 2019-12-08T13:52:42.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.3_2 2019-12-08T13:52:42.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.4 2019-12-08T13:52:42.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.4_1 2019-12-08T13:52:42.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.167+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.4_2 2019-12-08T13:52:42.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.368+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.5 2019-12-08T13:52:42.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.368+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.5_1 2019-12-08T13:52:42.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.380+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__shadow_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.5_2 2019-12-08T13:52:42.533 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.529+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.28__multipart_key1.2~S2InXjkFhQ9EGyDXOaqEEwFNf62EZ0C.6 2019-12-08T13:52:42.720 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.716+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:42.822 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.819+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:52:42.880 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.877+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:42.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.901+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.908+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:42.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:42.938+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:52:43.027 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.024+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:43.063 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.060+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:43.081 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.078+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:43.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.286+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:52:43.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.352+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:43.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.380+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:43.409 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:43.409 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.438', time=2019-12-08T15:14:31.233501+0000, chain.objs.size()=9 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.1 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.1_1 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.2 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.2_1 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.3 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.3_1 2019-12-08T13:52:43.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.4 2019-12-08T13:52:43.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.4_1 2019-12-08T13:52:43.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.406+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~feQ7d-gPyAp3SDsTws3j7ush30h-yiN.5 2019-12-08T13:52:43.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.653+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:43.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.794+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:52:43.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.830+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:43.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.846+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:43.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:43.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.666', time=2019-12-08T15:37:12.756378+0000, chain.objs.size()=9 2019-12-08T13:52:43.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__multipart_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.1 2019-12-08T13:52:43.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__shadow_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.1_1 2019-12-08T13:52:43.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__multipart_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.2 2019-12-08T13:52:43.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__shadow_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.2_1 2019-12-08T13:52:43.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__multipart_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.3 2019-12-08T13:52:43.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__shadow_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.3_1 2019-12-08T13:52:43.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__multipart_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.4 2019-12-08T13:52:43.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__shadow_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.4_1 2019-12-08T13:52:43.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:43.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.30__multipart_key1.2~2U3OdcovlcSkdvue3VbddrnjXNcvaOi.5 2019-12-08T13:52:44.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.103+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:44.188 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.184+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:52:44.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.227+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:44.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.242+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:44.260 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:44.260 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.509', time=2019-12-08T15:18:07.295847+0000, chain.objs.size()=9 2019-12-08T13:52:44.260 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__multipart_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.1 2019-12-08T13:52:44.260 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__shadow_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.1_1 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__multipart_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.2 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__shadow_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.2_1 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__multipart_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.3 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__shadow_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.3_1 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__multipart_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.4 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__shadow_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.4_1 2019-12-08T13:52:44.261 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.255+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.24__multipart_key1.2~1qOf-WMknHRJpQsdiHNA_7pD1XLV_Ld.5 2019-12-08T13:52:44.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.605+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:44.693 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.689+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:52:44.730 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.726+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:44.744 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.741+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:44.752 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.749+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:44.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.774+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:52:44.834 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.831+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:44.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.847+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:44.862 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.859+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:44.862 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.859+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~WQl0CKZwn43DpqLkCDdZqBAMpZyLMtL', time=2019-12-08T15:52:01.046284+0000, chain.objs.size()=0 2019-12-08T13:52:44.862 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.859+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:44.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:44.959+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:52:45.010 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.007+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:45.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.022+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:45.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:45.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-', time=2019-12-08T15:09:35.621078+0000, chain.objs.size()=9 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__multipart_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.1 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__shadow_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.1_1 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__multipart_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.2 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__shadow_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.2_1 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__multipart_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.3 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__shadow_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.3_1 2019-12-08T13:52:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__multipart_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.4 2019-12-08T13:52:45.043 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__shadow_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.4_1 2019-12-08T13:52:45.043 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.037+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.13__multipart_key1.2~JS4Gx_IH3BIwRhZ-y54ZJcjsqFyUBw-.5 2019-12-08T13:52:45.308 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.304+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:45.419 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.416+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:52:45.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.457+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:45.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.472+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:45.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:45.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.446', time=2019-12-08T15:15:11.444674+0000, chain.objs.size()=9 2019-12-08T13:52:45.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.1 2019-12-08T13:52:45.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.1_1 2019-12-08T13:52:45.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.2 2019-12-08T13:52:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.2_1 2019-12-08T13:52:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.3 2019-12-08T13:52:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.3_1 2019-12-08T13:52:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.4 2019-12-08T13:52:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.4_1 2019-12-08T13:52:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.485+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~Yo-W2zMXsZScNY8fZNBOF_Bqtq0kNYL.5 2019-12-08T13:52:45.726 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.722+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:45.835 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.830+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:52:45.879 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.874+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.896+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:45.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.905+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:45.950 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:45.945+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:52:46.012 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.008+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:46.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.046+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:46.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T13:52:46.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.466', time=2019-12-08T15:16:49.509776+0000, chain.objs.size()=7 2019-12-08T13:52:46.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.1 2019-12-08T13:52:46.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.1_1 2019-12-08T13:52:46.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.1_2 2019-12-08T13:52:46.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.2 2019-12-08T13:52:46.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.2_1 2019-12-08T13:52:46.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__shadow_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.2_2 2019-12-08T13:52:46.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.065+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.20__multipart_key1.2~4vZ5taoZebus_Cw3ifKe6ylDQFkaL2_.3 2019-12-08T13:52:46.334 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.330+0000 7f1ad3379ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T13:52:46.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.431+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:52:46.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.482+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:46.504 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.500+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:46.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.508+0000 7f1ad3379ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:46.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.584+0000 7f1ad3379ec0 20 remove_watcher() i=0 2019-12-08T13:52:46.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.584+0000 7f1ad3379ec0 2 removed watcher, disabling cache 2019-12-08T13:52:46.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.622+0000 7f1ad3379ec0 20 remove_watcher() i=1 2019-12-08T13:52:46.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.650+0000 7f1ad3379ec0 20 remove_watcher() i=2 2019-12-08T13:52:46.731 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.727+0000 7f1ad3379ec0 20 remove_watcher() i=3 2019-12-08T13:52:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.788+0000 7f1ad3379ec0 20 remove_watcher() i=4 2019-12-08T13:52:46.841 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.837+0000 7f1ad3379ec0 20 remove_watcher() i=5 2019-12-08T13:52:46.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.920+0000 7f1ad3379ec0 20 remove_watcher() i=6 2019-12-08T13:52:47.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:46.999+0000 7f1ad3379ec0 20 remove_watcher() i=7 2019-12-08T13:52:47.011 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:52:47.125 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.120+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.147 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.143+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.147 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.143+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.157+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.162 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.157+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.162+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.162+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.173 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.169+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.358+0000 7f47a8120ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:52:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.358+0000 7f47a8120ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:52:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.358+0000 7f47a8120ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:52:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.358+0000 7f47a8120ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:52:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.358+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.376 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.371+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:47.376 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.371+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.385+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:52:47.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.385+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.395+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:47.400 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.395+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.413+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:52:47.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.413+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.426 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.422+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.426 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.422+0000 7f47a8120ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:52:47.427 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.422+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.432 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.427+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.432 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.427+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.439+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:47.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.439+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.451+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:52:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.451+0000 7f47a8120ec0 20 zonegroup default 2019-12-08T13:52:47.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.451+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.456+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.456+0000 7f47a8120ec0 10 Cannot find current period zone using local zone 2019-12-08T13:52:47.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.456+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.466 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.462+0000 7f47a8120ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:47.467 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.462+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.471+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:47.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.471+0000 7f47a8120ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:47.484 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.479+0000 7f47a8120ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:52:47.484 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.479+0000 7f47a8120ec0 20 zone default found 2019-12-08T13:52:47.484 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.479+0000 7f47a8120ec0 20 started sync module instance, tier type = 2019-12-08T13:52:47.484 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.479+0000 7f47a8120ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:52:47.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=0 2019-12-08T13:52:47.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=1 2019-12-08T13:52:47.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=2 2019-12-08T13:52:47.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=3 2019-12-08T13:52:47.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=4 2019-12-08T13:52:47.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=5 2019-12-08T13:52:47.866 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.860+0000 7f47a8120ec0 20 add_watcher() i=6 2019-12-08T13:52:47.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.892+0000 7f47a8120ec0 20 add_watcher() i=7 2019-12-08T13:52:47.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.892+0000 7f47a8120ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:52:47.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.892+0000 7f4765fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:52:47.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.893+0000 7f4794b49700 0 -- >> v1:172.21.15.129:6789/0 conn(0x555e12b113a0 legacy=0x555e12b29b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-08T13:52:47.938 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:47.933+0000 7f4795b4b700 0 -- 172.21.15.129:0/3550470197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555e12ae7a00 msgr2=0x555e12b0e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:52:49.382 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.378+0000 7f46d7fff700 20 reqs_thread_entry: start 2019-12-08T13:52:49.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.378+0000 7f46d5ffb700 20 reqs_thread_entry: start 2019-12-08T13:52:49.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.379+0000 7f47a8120ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:52:49.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.379+0000 7f46c3fff700 20 reqs_thread_entry: start 2019-12-08T13:52:49.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.379+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:52:49.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.450+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.483 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.479+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:49.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.487+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.532 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.528+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:52:49.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.571+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.588+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:49.603 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.599+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.637+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:52:49.692 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.686+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.706 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.702+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:49.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.711+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.747+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:52:49.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.805+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.825 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.820+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:49.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.829+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.870+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:52:49.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.944+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:49.980 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.975+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:49.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:49.993+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.041+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:52:50.095 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.091+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.117 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.112+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:50.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.121+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.155+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:52:50.234 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.230+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.265 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.260+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:50.281 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.276+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.327 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.323+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:52:50.378 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.373+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.398+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:50.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.409+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.450+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:52:50.505 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.501+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.524 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.520+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:50.533 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.529+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.566 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.561+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:52:50.641 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.637+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.670 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.666+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:50.684 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.680+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.745 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.739+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:52:50.840 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.835+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.867+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:50.895 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.891+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:50.952 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:50.947+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:52:51.037 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.033+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.061+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.078 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.074+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.136 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.131+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:52:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.213+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.251 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.247+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.264+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.316+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:52:51.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.351+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.365+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.378 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.373+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.401+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:52:51.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.446+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.465 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.460+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.476 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.472+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.536+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:52:51.604 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.600+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.640 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.635+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.655+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.698+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:52:51.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.741+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.761 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.757+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.769 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.765+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.805+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:52:51.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.873+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.911+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:51.930 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.926+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:51.993 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:51.989+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:52:52.061 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.057+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.080+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.096 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.092+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.148+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:52:52.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.219+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.249+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.263+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.337+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:52:52.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.406+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.429 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.424+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.434+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.483 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.478+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:52:52.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.525+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.540+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.548+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.593+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:52:52.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.631+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.645+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.653+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.694 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.690+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:52:52.738 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.733+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.748+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.762 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.757+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.802+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:52:52.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.845+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.861+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:52.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.872+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:52.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.899+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:52:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:52.983+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.017+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.037+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.094 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.091+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:52:53.173 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.170+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.217 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.213+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.244 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.241+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.293 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.290+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:52:53.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.348+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.369+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.380+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.427+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:52:53.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.485+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.513+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.526+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.563+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:52:53.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.606+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.624 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.620+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.630+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.670+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:52:53.741 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.737+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.762 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.758+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.775 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.772+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.831 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.828+0000 7f47a8120ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:52:53.885 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.882+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:53.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.897+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:53.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:53.906+0000 7f47a8120ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:54.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.012+0000 7f47a8120ec0 20 remove_watcher() i=0 2019-12-08T13:52:54.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.012+0000 7f47a8120ec0 2 removed watcher, disabling cache 2019-12-08T13:52:54.053 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.050+0000 7f47a8120ec0 20 remove_watcher() i=1 2019-12-08T13:52:54.089 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.086+0000 7f47a8120ec0 20 remove_watcher() i=2 2019-12-08T13:52:54.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.149+0000 7f47a8120ec0 20 remove_watcher() i=3 2019-12-08T13:52:54.203 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.200+0000 7f47a8120ec0 20 remove_watcher() i=4 2019-12-08T13:52:54.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.242+0000 7f47a8120ec0 20 remove_watcher() i=5 2019-12-08T13:52:54.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.314+0000 7f47a8120ec0 20 remove_watcher() i=6 2019-12-08T13:52:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.364+0000 7f47a8120ec0 20 remove_watcher() i=7 2019-12-08T13:52:54.377 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:52:54.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.500+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.525 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.522+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.525 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.522+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.531+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.534 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.531+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.536+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.536+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.547+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:54.722 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.719+0000 7fa439756ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:52:54.722 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.719+0000 7fa439756ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:52:54.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.719+0000 7fa439756ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:52:54.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.719+0000 7fa439756ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:52:54.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.719+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.730 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.727+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:54.730 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.727+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.738 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.734+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:52:54.738 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.734+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.748+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:54.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.748+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.768 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.765+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:52:54.768 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.765+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.769+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.769+0000 7fa439756ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:52:54.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.770+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.779 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.776+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.779 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.776+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.788 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.785+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:54.788 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.785+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.794+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:52:54.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.794+0000 7fa439756ec0 20 zonegroup default 2019-12-08T13:52:54.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.794+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.802 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.798+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.802 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.798+0000 7fa439756ec0 10 Cannot find current period zone using local zone 2019-12-08T13:52:54.802 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.798+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.807 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.804+0000 7fa439756ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:52:54.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.804+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.811+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:52:54.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.811+0000 7fa439756ec0 20 rados->read ofs=0 len=0 2019-12-08T13:52:54.822 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.818+0000 7fa439756ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:52:54.822 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.818+0000 7fa439756ec0 20 zone default found 2019-12-08T13:52:54.822 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.818+0000 7fa439756ec0 20 started sync module instance, tier type = 2019-12-08T13:52:54.823 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:54.818+0000 7fa439756ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:52:55.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.219+0000 7fa439756ec0 20 add_watcher() i=0 2019-12-08T13:52:55.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.220+0000 7fa439756ec0 20 add_watcher() i=1 2019-12-08T13:52:55.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.220+0000 7fa439756ec0 20 add_watcher() i=2 2019-12-08T13:52:55.224 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.220+0000 7fa439756ec0 20 add_watcher() i=3 2019-12-08T13:52:55.224 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.220+0000 7fa439756ec0 20 add_watcher() i=4 2019-12-08T13:52:55.224 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.220+0000 7fa439756ec0 20 add_watcher() i=5 2019-12-08T13:52:55.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.242+0000 7fa439756ec0 20 add_watcher() i=6 2019-12-08T13:52:55.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.255+0000 7fa439756ec0 20 add_watcher() i=7 2019-12-08T13:52:55.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.255+0000 7fa439756ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:52:55.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.255+0000 7fa3f77de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:52:55.270 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.266+0000 7fa427181700 0 -- 172.21.15.129:0/240731709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa368002850 msgr2=0x7fa368002e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-08T13:52:55.328 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:55.325+0000 7fa41ffff700 0 -- 172.21.15.129:0/3787043119 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x5615e166f4e0 msgr2=0x5615e166f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:52:56.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.602+0000 7fa365ffb700 20 reqs_thread_entry: start 2019-12-08T13:52:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.603+0000 7fa35f7fe700 20 reqs_thread_entry: start 2019-12-08T13:52:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.603+0000 7fa439756ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:52:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.603+0000 7fa35dffb700 20 reqs_thread_entry: start 2019-12-08T13:52:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.603+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:52:56.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.663+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:56.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.682+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:56.696 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.693+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:56.741 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.738+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:52:56.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.816+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:56.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.847+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:56.866 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.863+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:56.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.906+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:52:56.946 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.943+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:56.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.958+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:56.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:56.967+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.007 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.004+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:52:57.121 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.117+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.151+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:57.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.162+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.216 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.213+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:52:57.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.253+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.271 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.268+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:57.281 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.278+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.307+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:52:57.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.349+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.364+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:57.378 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.374+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.403+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:52:57.485 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.481+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.511+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:57.527 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.524+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.597 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.594+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:52:57.679 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.676+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.707+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.725+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.800 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.797+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:52:57.867 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.863+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.885+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:57.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.894+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:57.931+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:52:58.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.041+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.075 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.072+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.087 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.084+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.149+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:52:58.195 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.191+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.213 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.209+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.222 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.219+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.249+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:52:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.292+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.307+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.315+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.353+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:52:58.443 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.439+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.461+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.477 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.473+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.513+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:52:58.565 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.562+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.584+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.594+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.639+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:52:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.704+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.738 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.735+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.743+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.794+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:52:58.851 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.847+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.869 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.866+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:58.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.874+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:58.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.914+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:52:58.995 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:58.992+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.024 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.020+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:59.038 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.034+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.079+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:52:59.141 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.138+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.163 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.160+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.169+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.348+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:52:59.424 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.420+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.458+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:59.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.466+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.546+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:52:59.596 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.593+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.614+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:59.628 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.625+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.656+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:52:59.710 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.706+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.724+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:59.738 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.735+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.772 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.769+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:52:59.824 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.821+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.842 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.839+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:52:59.851 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.847+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.880+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:52:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.958+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:52:59.989 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:52:59.985+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.005 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.002+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.065 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.061+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:53:00.133 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.129+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.157 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.153+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.167+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.211+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:53:00.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.294+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.309+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.320+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.352+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:53:00.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.396+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.419+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.431 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.428+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.471+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:53:00.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.521+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.537+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.545+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.575+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:53:00.641 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.638+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.653+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.664+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.721 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.717+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:53:00.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.777+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.802 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.798+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.811+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.870 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.867+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:53:00.927 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.923+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.939+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:00.951 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.948+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:00.994 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:00.990+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:53:01.033 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.029+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:01.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.044+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:01.055 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.052+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:01.105 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.102+0000 7fa439756ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:53:01.142 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.138+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:01.157 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.154+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:01.167 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.163+0000 7fa439756ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:01.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.269+0000 7fa439756ec0 20 remove_watcher() i=0 2019-12-08T13:53:01.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.269+0000 7fa439756ec0 2 removed watcher, disabling cache 2019-12-08T13:53:01.302 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.299+0000 7fa439756ec0 20 remove_watcher() i=1 2019-12-08T13:53:01.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.329+0000 7fa439756ec0 20 remove_watcher() i=2 2019-12-08T13:53:01.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.381+0000 7fa439756ec0 20 remove_watcher() i=3 2019-12-08T13:53:01.437 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.434+0000 7fa439756ec0 20 remove_watcher() i=4 2019-12-08T13:53:01.484 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.481+0000 7fa439756ec0 20 remove_watcher() i=5 2019-12-08T13:53:01.536 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.532+0000 7fa439756ec0 20 remove_watcher() i=6 2019-12-08T13:53:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.582+0000 7fa439756ec0 20 remove_watcher() i=7 2019-12-08T13:53:01.596 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:53:01.720 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.717+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.758 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.754+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:01.758 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.754+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.763+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:01.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.763+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.769+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:01.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.770+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.787 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.783+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:53:01.960 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.956+0000 7f7e8c7e5ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:53:01.960 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.956+0000 7f7e8c7e5ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:53:01.960 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.956+0000 7f7e8c7e5ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:53:01.960 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.956+0000 7f7e8c7e5ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:53:01.960 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.956+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.965+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:53:01.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.965+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.978 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.975+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:53:01.979 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.975+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:01.988 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.985+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:53:01.988 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.985+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.000 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.997+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:53:02.001 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:01.997+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.005 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.002+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:02.005 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.002+0000 7f7e8c7e5ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:53:02.005 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.002+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.012 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.009+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:02.012 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.009+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.033 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.022+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:53:02.033 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.022+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.037+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:53:02.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.037+0000 7f7e8c7e5ec0 20 zonegroup default 2019-12-08T13:53:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.037+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.042+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:02.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.042+0000 7f7e8c7e5ec0 10 Cannot find current period zone using local zone 2019-12-08T13:53:02.046 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.042+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.050 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.047+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:53:02.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.047+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.054+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:53:02.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.054+0000 7f7e8c7e5ec0 20 rados->read ofs=0 len=0 2019-12-08T13:53:02.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.065+0000 7f7e8c7e5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:53:02.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.065+0000 7f7e8c7e5ec0 20 zone default found 2019-12-08T13:53:02.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.065+0000 7f7e8c7e5ec0 20 started sync module instance, tier type = 2019-12-08T13:53:02.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.065+0000 7f7e8c7e5ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:53:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=0 2019-12-08T13:53:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=1 2019-12-08T13:53:02.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=2 2019-12-08T13:53:02.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=3 2019-12-08T13:53:02.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=4 2019-12-08T13:53:02.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=5 2019-12-08T13:53:02.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.418+0000 7f7e8c7e5ec0 20 add_watcher() i=6 2019-12-08T13:53:02.446 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.443+0000 7f7e8c7e5ec0 20 add_watcher() i=7 2019-12-08T13:53:02.447 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.443+0000 7f7e8c7e5ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:53:02.447 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.443+0000 7f7e4a7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:53:02.450 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.446+0000 7f7e79a0f700 0 -- 172.21.15.129:0/2391173107 >> v2:172.21.15.129:3300/0 conn(0x55a27ad31c80 msgr2=0x55a27ad32090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T13:53:02.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:02.486+0000 7f7e79a0f700 0 -- 172.21.15.129:0/3446182617 >> [v2:172.21.15.129:6810/10196,v1:172.21.15.129:6811/10196] conn(0x55a27ad68350 msgr2=0x55a27ad6a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-08T13:53:04.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.366+0000 7f7dc0ff9700 20 reqs_thread_entry: start 2019-12-08T13:53:04.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.366+0000 7f7dba7fc700 20 reqs_thread_entry: start 2019-12-08T13:53:04.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.366+0000 7f7e8c7e5ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:53:04.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.367+0000 7f7db8ff9700 20 reqs_thread_entry: start 2019-12-08T13:53:04.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.367+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:53:04.420 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.417+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.439 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.436+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:04.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.446+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.493+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:53:04.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.533+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.563 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.550+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:04.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.559+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.585+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:53:04.652 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.649+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.679+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:04.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.693+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.748 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.745+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:53:04.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.825+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.846 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.842+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:04.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.851+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.888 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.885+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:53:04.930 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.927+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.941+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:04.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.950+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:04.984 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:04.981+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:53:05.060 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.056+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.091 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.085+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:05.111 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.107+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.167+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:53:05.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.254+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.291 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.288+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:05.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.308+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.366+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:53:05.435 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.427+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.459 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.455+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:05.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.465+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.506+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:53:05.584 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.580+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.611 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.608+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:05.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.624+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.679 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.676+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:53:05.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.734+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.751+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:05.763 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.759+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.793+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:53:05.856 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.853+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.870+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:05.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.879+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:05.932 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:05.929+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:53:06.028 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.024+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.057 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.054+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.072 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.069+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.120+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:53:06.219 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.216+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.252+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.262+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.301+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:53:06.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.352+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.378 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.374+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.386+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.445 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.441+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:53:06.491 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.488+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.508+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.521 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.517+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.549+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.630+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.655+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.669+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.715 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.711+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:53:06.766 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.762+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.778+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.787+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.857+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:53:06.917 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.914+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.933 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.930+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:06.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.938+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:06.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:06.982+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:53:07.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.041+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.059 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.055+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.072 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.068+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.106 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.102+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:53:07.141 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.138+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.156 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.153+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.161+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.200 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.197+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.236+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.251+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.261+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.296+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:53:07.374 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.371+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.398+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.419+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.476 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.472+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:53:07.556 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.553+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.582+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.603 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.599+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.650+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:53:07.714 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.710+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.730+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.744 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.741+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.790 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.786+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:53:07.837 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.834+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.856 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.852+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.861+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.898+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:53:07.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.940+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:07.960 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.957+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:07.972 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:07.969+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.007+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:53:08.057 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.053+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.072 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.068+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:08.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.078+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.112+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:53:08.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.167+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.187 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.184+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:08.197 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.194+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.243 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.239+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:53:08.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.293+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.315 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.312+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:08.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.320+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.369+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:53:08.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.414+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.432 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.429+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:08.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.438+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.482 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.479+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:53:08.529 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.526+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.546 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.543+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:08.554 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.550+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.585 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.582+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:53:08.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.640+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.658+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:53:08.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.670+0000 7f7e8c7e5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:53:08.784 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.781+0000 7f7e8c7e5ec0 20 remove_watcher() i=0 2019-12-08T13:53:08.784 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.781+0000 7f7e8c7e5ec0 2 removed watcher, disabling cache 2019-12-08T13:53:08.822 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.819+0000 7f7e8c7e5ec0 20 remove_watcher() i=1 2019-12-08T13:53:08.855 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.852+0000 7f7e8c7e5ec0 20 remove_watcher() i=2 2019-12-08T13:53:08.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.917+0000 7f7e8c7e5ec0 20 remove_watcher() i=3 2019-12-08T13:53:08.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:08.955+0000 7f7e8c7e5ec0 20 remove_watcher() i=4 2019-12-08T13:53:09.012 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:09.009+0000 7f7e8c7e5ec0 20 remove_watcher() i=5 2019-12-08T13:53:09.068 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:09.064+0000 7f7e8c7e5ec0 20 remove_watcher() i=6 2019-12-08T13:53:09.131 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:53:09.128+0000 7f7e8c7e5ec0 20 remove_watcher() i=7 2019-12-08T13:53:09.141 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-12-08T13:53:09.766 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :clean 2019-12-08T13:53:10.463 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :compileJava 2019-12-08T13:53:10.464 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :processResources 2019-12-08T13:53:10.464 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :classes 2019-12-08T13:53:10.763 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:10.764 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :compileTestJava 2019-12-08T13:53:10.764 INFO:teuthology.orchestra.run.smithi129.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-08T13:53:10.764 INFO:teuthology.orchestra.run.smithi129.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-08T13:53:10.764 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:10.765 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :processTestResources NO-SOURCE 2019-12-08T13:53:10.765 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :testClasses 2019-12-08T13:53:19.863 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:19.864 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :test 2019-12-08T13:53:19.864 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:19.864 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-12-08T13:53:22.063 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:22.063 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-12-08T13:53:36.863 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:36.863 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-12-08T13:53:39.763 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:39.763 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-12-08T13:53:45.963 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:45.963 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-12-08T13:53:48.063 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:48.063 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-12-08T13:53:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:53:54.063 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-12-08T13:54:00.363 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:00.363 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-12-08T13:54:01.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:01.563 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-12-08T13:54:06.263 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:06.263 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-12-08T13:54:14.663 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:14.663 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-12-08T13:54:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:15.563 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-12-08T13:54:17.163 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:17.163 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-12-08T13:54:23.263 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:23.263 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-12-08T13:54:37.463 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:37.463 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-12-08T13:54:39.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:39.563 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-12-08T13:54:46.563 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:46.563 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-12-08T13:54:46.863 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:46.863 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-12-08T13:54:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:48.199 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------- 2019-12-08T13:54:48.199 INFO:teuthology.orchestra.run.smithi129.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-12-08T13:54:48.199 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------- 2019-12-08T13:54:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:54:48.199 INFO:teuthology.orchestra.run.smithi129.stdout:BUILD SUCCESSFUL in 1m 38s 2019-12-08T13:54:48.199 INFO:teuthology.orchestra.run.smithi129.stdout:5 actionable tasks: 5 executed 2019-12-08T13:54:48.203 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:54:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.354+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.582+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.582+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.592+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.596 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.592+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.599+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.599+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.613 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.610+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:48.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.804+0000 7fb60d64bec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:54:48.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.804+0000 7fb60d64bec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:54:48.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.804+0000 7fb60d64bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:54:48.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.804+0000 7fb60d64bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:54:48.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.804+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.823 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.819+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:48.823 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.819+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.830+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:54:48.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.830+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.842 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.839+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:48.842 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.839+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.858+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:54:48.862 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.858+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.864+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.864+0000 7fb60d64bec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:54:48.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.864+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.871+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.871+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.882+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:48.886 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.882+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.899+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:54:48.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.899+0000 7fb60d64bec0 20 zonegroup default 2019-12-08T13:54:48.903 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.899+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.905+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.905+0000 7fb60d64bec0 10 Cannot find current period zone using local zone 2019-12-08T13:54:48.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.905+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.910+0000 7fb60d64bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:48.914 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.910+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.920+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:48.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.920+0000 7fb60d64bec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:48.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.930+0000 7fb60d64bec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:54:48.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.930+0000 7fb60d64bec0 20 zone default found 2019-12-08T13:54:48.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.930+0000 7fb60d64bec0 20 started sync module instance, tier type = 2019-12-08T13:54:48.934 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:48.930+0000 7fb60d64bec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:54:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=0 2019-12-08T13:54:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=1 2019-12-08T13:54:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=2 2019-12-08T13:54:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=3 2019-12-08T13:54:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=4 2019-12-08T13:54:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=5 2019-12-08T13:54:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.278+0000 7fb60d64bec0 20 add_watcher() i=6 2019-12-08T13:54:49.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.317+0000 7fb60d64bec0 20 add_watcher() i=7 2019-12-08T13:54:49.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.317+0000 7fb60d64bec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:54:49.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.317+0000 7fb5cb7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:54:49.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.321+0000 7fb5fa875700 0 -- 172.21.15.129:0/2380872873 >> v1:172.21.15.129:6789/0 conn(0x557c258c2b60 legacy=0x557c258c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:54:49.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:49.364+0000 7fb5fa074700 0 -- 172.21.15.129:0/3859820074 >> [v2:172.21.15.129:6810/10196,v1:172.21.15.129:6811/10196] conn(0x557c258f8d60 msgr2=0x557c258fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:54:50.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:50.955+0000 7fb539ffb700 20 reqs_thread_entry: start 2019-12-08T13:54:50.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:50.955+0000 7fb5337fe700 20 reqs_thread_entry: start 2019-12-08T13:54:50.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:50.955+0000 7fb60d64bec0 20 init_complete bucket index max shards: 0 2019-12-08T13:54:50.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:50.956+0000 7fb531ffb700 20 reqs_thread_entry: start 2019-12-08T13:54:50.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:50.956+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:54:51.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.022+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.039+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.048+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.097 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.094+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:54:51.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.182+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.212 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.208+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.224+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.274 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.270+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:54:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.331+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.357+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.392 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.388+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.429+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:54:51.500 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.496+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.530 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.527+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.546+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.610 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.607+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:54:51.669 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.665+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.691 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.687+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.700+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.743+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:54:51.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.803+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.820 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.816+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.825+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.861+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:54:51.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.901+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.915+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:51.926 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.923+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:51.963 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:51.960+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:54:52.013 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.009+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.029 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.026+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.039 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.036+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.066+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:54:52.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.125+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.156 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.153+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.166+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.198+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:54:52.285 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.282+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.319 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.316+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.333+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.385+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:54:52.460 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.457+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.486+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.502 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.498+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.546 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.542+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:54:52.602 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.598+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.620 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.616+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.633 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.629+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.691 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.688+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:54:52.760 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.757+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.779+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.790 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.787+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.834 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.831+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:54:52.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.873+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.892 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.889+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:52.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.897+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:52.932 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.928+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:54:52.994 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:52.990+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.013 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.010+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.021+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.076 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.072+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:54:53.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.140+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.157+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.169 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.166+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.220 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.217+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:54:53.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.280+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.298+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.306+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.356+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:54:53.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.412+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.427+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.438 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.435+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.467 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.464+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:54:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.513+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.531 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.527+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.539 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.535+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.580 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.576+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:54:53.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.640+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.661+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.675 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.672+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.726 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.723+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:54:53.785 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.782+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.807+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.834 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.831+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.880 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.876+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:54:53.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.921+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:53.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.945+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:53.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:53.955+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.000+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:54:54.086 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.083+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.123+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:54.146 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.142+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.218 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.215+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:54:54.259 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.256+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.274+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:54.286 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.283+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.313+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:54:54.354 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.350+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.368+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.376+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.410+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:54:54.505 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.502+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.537 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.533+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:54.551 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.548+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.608 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.605+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:54:54.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.682+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.708+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:54.730 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.727+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.773 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.770+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:54:54.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.865+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.900 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.897+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:54.913 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.909+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:54.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:54.960+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:54:55.036 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.033+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.060 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.056+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.071+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.113+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:54:55.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.162+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.186 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.183+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:55.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.191+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.228+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:54:55.295 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.291+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.307+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:55.320 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.317+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.369 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.365+0000 7fb60d64bec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:54:55.500 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.496+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.526 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.523+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:55.539 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.535+0000 7fb60d64bec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:55.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.624+0000 7fb60d64bec0 20 remove_watcher() i=0 2019-12-08T13:54:55.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.624+0000 7fb60d64bec0 2 removed watcher, disabling cache 2019-12-08T13:54:55.660 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.656+0000 7fb60d64bec0 20 remove_watcher() i=1 2019-12-08T13:54:55.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.697+0000 7fb60d64bec0 20 remove_watcher() i=2 2019-12-08T13:54:55.764 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.760+0000 7fb60d64bec0 20 remove_watcher() i=3 2019-12-08T13:54:55.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.803+0000 7fb60d64bec0 20 remove_watcher() i=4 2019-12-08T13:54:55.866 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.863+0000 7fb60d64bec0 20 remove_watcher() i=5 2019-12-08T13:54:55.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.899+0000 7fb60d64bec0 20 remove_watcher() i=6 2019-12-08T13:54:55.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:55.941+0000 7fb60d64bec0 20 remove_watcher() i=7 2019-12-08T13:54:55.952 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:54:56.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.053+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.080 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.076+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.080 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.076+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.092 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.089+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.093 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.089+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.096+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.096+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.103+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:56.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.310+0000 7fd733963ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:54:56.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.310+0000 7fd733963ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:54:56.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.310+0000 7fd733963ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:54:56.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.310+0000 7fd733963ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:54:56.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.310+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.319+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:56.323 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.319+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.330 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.326+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:54:56.330 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.326+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.336+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:56.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.337+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.346+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:54:56.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.346+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.354 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.351+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.351+0000 7fd733963ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:54:56.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.351+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.360 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.356+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.360 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.356+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.363+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:56.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.364+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.375 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.372+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:54:56.375 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.372+0000 7fd733963ec0 20 zonegroup default 2019-12-08T13:54:56.375 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.372+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.376+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.376+0000 7fd733963ec0 10 Cannot find current period zone using local zone 2019-12-08T13:54:56.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.376+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.381+0000 7fd733963ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:54:56.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.381+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.393 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.389+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:54:56.393 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.389+0000 7fd733963ec0 20 rados->read ofs=0 len=0 2019-12-08T13:54:56.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.398+0000 7fd733963ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:54:56.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.398+0000 7fd733963ec0 20 zone default found 2019-12-08T13:54:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.398+0000 7fd733963ec0 20 started sync module instance, tier type = 2019-12-08T13:54:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.398+0000 7fd733963ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:54:56.832 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.829+0000 7fd733963ec0 20 add_watcher() i=0 2019-12-08T13:54:56.832 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.829+0000 7fd733963ec0 20 add_watcher() i=1 2019-12-08T13:54:56.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.829+0000 7fd733963ec0 20 add_watcher() i=2 2019-12-08T13:54:56.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.829+0000 7fd733963ec0 20 add_watcher() i=3 2019-12-08T13:54:56.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.829+0000 7fd733963ec0 20 add_watcher() i=4 2019-12-08T13:54:56.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.829+0000 7fd733963ec0 20 add_watcher() i=5 2019-12-08T13:54:56.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.858+0000 7fd733963ec0 20 add_watcher() i=6 2019-12-08T13:54:56.872 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.868+0000 7fd733963ec0 20 add_watcher() i=7 2019-12-08T13:54:56.872 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.868+0000 7fd733963ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:54:56.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.869+0000 7fd6f0fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:54:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.890+0000 7fd72138e700 0 -- 172.21.15.129:0/2852039851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c499d0e9f0 msgr2=0x55c499d22a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:54:56.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:56.918+0000 7fd72138e700 0 -- 172.21.15.129:0/464438475 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x55c499d3af90 msgr2=0x55c499d3a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:54:58.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.398+0000 7fd662ffd700 20 reqs_thread_entry: start 2019-12-08T13:54:58.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.398+0000 7fd660ff9700 20 reqs_thread_entry: start 2019-12-08T13:54:58.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.398+0000 7fd733963ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:54:58.404 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.398+0000 7fd656ffd700 20 reqs_thread_entry: start 2019-12-08T13:54:58.404 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.399+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:54:58.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.453+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:58.505 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.500+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:58.515 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.510+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:58.552 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.548+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:54:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.615+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:58.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.637+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:58.651 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.647+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:58.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.724+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:54:58.810 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.806+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:58.841 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.837+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:58.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.850+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:58.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.914+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:54:58.984 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:58.980+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.050 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.046+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:59.060 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.056+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.114 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.109+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:54:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.185+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.223 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.218+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:59.237 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.233+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.308 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.303+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:54:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.351+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.374 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.370+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:59.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.380+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.466 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.461+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:54:59.516 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.511+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.532 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.527+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:59.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.536+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.578 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.573+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:54:59.621 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.617+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.633+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:59.646 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.642+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.680 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.675+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:54:59.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.775+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.812 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.807+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:54:59.831 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.827+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:54:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.892+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:54:59.983 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:54:59.979+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.011+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.030 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.025+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.079+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:55:00.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.140+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.160+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.178 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.174+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.221 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.216+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:55:00.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.284+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.305+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.321 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.317+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.364+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:55:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.407+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.425+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.435+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.468+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:55:00.521 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.516+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.537 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.533+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.540+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.586+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:55:00.636 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.632+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.651+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.660+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.710 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.705+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:55:00.784 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.780+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.799+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.812 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.807+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.850+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:55:00.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.893+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.919 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.914+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:00.927 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.923+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:00.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.954+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:55:01.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:00.999+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.017+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.033 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.028+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.077 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.073+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:55:01.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.123+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.145 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.141+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.153 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.149+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.210 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.205+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:55:01.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.282+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.312+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.331+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.408 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.404+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:55:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.470+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.490+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.506 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.501+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.537+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:55:01.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.613+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.652 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.648+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.670+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.724+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:55:01.789 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.785+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.807+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.824 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.819+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.859+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:55:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.915+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.935 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.931+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:01.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.939+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:01.978 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:01.973+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:55:02.063 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.058+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.094 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.089+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.114 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.110+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.156+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:55:02.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.223+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.252 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.247+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.262 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.257+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.316 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.312+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:55:02.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.378+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.399+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.416 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.411+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.459+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:55:02.526 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.522+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.551 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.547+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.569 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.565+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.613 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.609+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:55:02.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.657+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.677 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.673+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.681+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.707+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:55:02.760 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.756+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.784 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.779+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.796 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.792+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.835+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:55:02.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.918+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:02.955 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.951+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:02.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:02.966+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:03.026 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.021+0000 7fd733963ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:55:03.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.095+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:03.121 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.117+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:03.132 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.128+0000 7fd733963ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.223+0000 7fd733963ec0 20 remove_watcher() i=0 2019-12-08T13:55:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.223+0000 7fd733963ec0 2 removed watcher, disabling cache 2019-12-08T13:55:03.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.285+0000 7fd733963ec0 20 remove_watcher() i=1 2019-12-08T13:55:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.329+0000 7fd733963ec0 20 remove_watcher() i=2 2019-12-08T13:55:03.374 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.370+0000 7fd733963ec0 20 remove_watcher() i=3 2019-12-08T13:55:03.425 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.421+0000 7fd733963ec0 20 remove_watcher() i=4 2019-12-08T13:55:03.489 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.484+0000 7fd733963ec0 20 remove_watcher() i=5 2019-12-08T13:55:03.556 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.551+0000 7fd733963ec0 20 remove_watcher() i=6 2019-12-08T13:55:03.628 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.624+0000 7fd733963ec0 20 remove_watcher() i=7 2019-12-08T13:55:03.641 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:55:03.804 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.800+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:03.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.825+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:03.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.825+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:03.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.835+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:03.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.835+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:03.844 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.840+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:03.844 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.840+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:03.852 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:03.848+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:04.057 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.052+0000 7f4905681ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:55:04.057 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.052+0000 7f4905681ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:55:04.057 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.052+0000 7f4905681ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:55:04.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.052+0000 7f4905681ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:55:04.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.053+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.071 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.066+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:04.071 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.066+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.079+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:55:04.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.079+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.095 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.091+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:04.095 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.091+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.115 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.110+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:55:04.115 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.110+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.120 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.116+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:04.120 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.116+0000 7f4905681ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:55:04.120 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.116+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.122+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:04.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.122+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.137 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.132+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:04.137 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.132+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.155+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:55:04.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.155+0000 7f4905681ec0 20 zonegroup default 2019-12-08T13:55:04.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.155+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.161+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:04.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.161+0000 7f4905681ec0 10 Cannot find current period zone using local zone 2019-12-08T13:55:04.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.161+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.170+0000 7f4905681ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:04.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.170+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.182 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.178+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:04.182 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.178+0000 7f4905681ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:04.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.187+0000 7f4905681ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:55:04.192 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.187+0000 7f4905681ec0 20 zone default found 2019-12-08T13:55:04.192 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.187+0000 7f4905681ec0 20 started sync module instance, tier type = 2019-12-08T13:55:04.192 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.187+0000 7f4905681ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:55:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.582+0000 7f4905681ec0 20 add_watcher() i=0 2019-12-08T13:55:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.582+0000 7f4905681ec0 20 add_watcher() i=1 2019-12-08T13:55:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.582+0000 7f4905681ec0 20 add_watcher() i=2 2019-12-08T13:55:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.582+0000 7f4905681ec0 20 add_watcher() i=3 2019-12-08T13:55:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.582+0000 7f4905681ec0 20 add_watcher() i=4 2019-12-08T13:55:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.582+0000 7f4905681ec0 20 add_watcher() i=5 2019-12-08T13:55:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.589+0000 7f4905681ec0 20 add_watcher() i=6 2019-12-08T13:55:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.621+0000 7f4905681ec0 20 add_watcher() i=7 2019-12-08T13:55:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.621+0000 7f4905681ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:55:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.621+0000 7f48c37de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:55:04.631 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.626+0000 7f48f20aa700 0 -- 172.21.15.129:0/1187741472 >> v1:172.21.15.181:6789/0 conn(0x55b2087b4500 legacy=0x55b2087b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:55:04.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:04.663+0000 7f48f20aa700 0 -- 172.21.15.129:0/1888406318 >> [v2:172.21.15.129:6810/10196,v1:172.21.15.129:6811/10196] conn(0x55b2087e0c30 msgr2=0x55b2087e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:55:06.401 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.397+0000 7f4831ffb700 20 reqs_thread_entry: start 2019-12-08T13:55:06.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.398+0000 7f482b7fe700 20 reqs_thread_entry: start 2019-12-08T13:55:06.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.398+0000 7f4905681ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:55:06.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.398+0000 7f4829ffb700 20 reqs_thread_entry: start 2019-12-08T13:55:06.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.398+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:55:06.456 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.451+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.470 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.466+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:06.478 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.474+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.506+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:55:06.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.590+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.623+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:06.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.634+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.679+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:55:06.759 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.755+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.784 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.781+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:06.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.795+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.854 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.850+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:55:06.931 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.928+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:06.955 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.952+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:06.967 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:06.964+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.018 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.015+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:55:07.105 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.101+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.140 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.136+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:07.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.149+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.208 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.204+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:55:07.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.263+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.296+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:07.308 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.304+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.332+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:55:07.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.414+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.451+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:07.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.466+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.655+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:55:07.712 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.709+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.731 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.728+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:07.752 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.749+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.810+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:55:07.869 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.866+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.895 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.891+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:07.908 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.904+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:07.975 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:07.972+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:55:08.039 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.036+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.060 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.057+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.066+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.228+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:55:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.307+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.327 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.323+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.334+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.379+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:55:08.427 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.424+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.440+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.450+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.496+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:55:08.572 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.569+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.587+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.601 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.598+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.633+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:55:08.688 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.684+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.720+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.743 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.740+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.800 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.794+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:55:08.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.846+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.871 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.867+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.879+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.917+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:55:08.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.958+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:08.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.974+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:08.986 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:08.983+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.020 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.017+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:55:09.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.063+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.078+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.091 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.087+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.124+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:55:09.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.191+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.211 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.208+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.220 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.217+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.265+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:55:09.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.325+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.342+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.354 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.351+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.408 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.404+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:55:09.470 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.466+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.490 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.487+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.502 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.498+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.550 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.547+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:55:09.608 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.605+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.623 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.620+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.628+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.661+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:55:09.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.700+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.720 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.717+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.731 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.728+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.757 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.754+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:55:09.846 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.842+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.871 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.868+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:09.880 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.877+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:09.932 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:09.929+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:55:10.022 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.018+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.054 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.050+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.066+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.121 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.117+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:55:10.173 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.170+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.188+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.202 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.198+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.227+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:55:10.286 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.282+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.329 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.326+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.338+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.380+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:55:10.432 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.428+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.447+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.457+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.488+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:55:10.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.550+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.569 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.565+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.577 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.574+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.603+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:55:10.679 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.675+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.709 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.705+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.716+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.762+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:55:10.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.825+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.871 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.867+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:10.885 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.882+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:10.947 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.944+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:55:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.983+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:11.001 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:10.998+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:11.011 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.007+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.037+0000 7f4905681ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:55:11.079 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.075+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:11.095 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.091+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:11.117 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.113+0000 7f4905681ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:11.208 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.205+0000 7f4905681ec0 20 remove_watcher() i=0 2019-12-08T13:55:11.209 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.205+0000 7f4905681ec0 2 removed watcher, disabling cache 2019-12-08T13:55:11.243 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.239+0000 7f4905681ec0 20 remove_watcher() i=1 2019-12-08T13:55:11.278 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.274+0000 7f4905681ec0 20 remove_watcher() i=2 2019-12-08T13:55:11.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.357+0000 7f4905681ec0 20 remove_watcher() i=3 2019-12-08T13:55:11.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.424+0000 7f4905681ec0 20 remove_watcher() i=4 2019-12-08T13:55:11.466 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.462+0000 7f4905681ec0 20 remove_watcher() i=5 2019-12-08T13:55:11.508 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.505+0000 7f4905681ec0 20 remove_watcher() i=6 2019-12-08T13:55:11.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.550+0000 7f4905681ec0 20 remove_watcher() i=7 2019-12-08T13:55:11.561 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T13:55:11.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.663+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.690 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.686+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.690 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.686+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.694+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.694+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.702 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.699+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.703 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.699+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.706+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:11.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.870+0000 7f4e388b8ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T13:55:11.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.870+0000 7f4e388b8ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T13:55:11.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.870+0000 7f4e388b8ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T13:55:11.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.870+0000 7f4e388b8ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T13:55:11.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.870+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.881 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.878+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:11.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.878+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.885+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:55:11.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.885+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.892+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:11.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.892+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.910 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.906+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:55:11.910 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.906+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.913+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.913+0000 7f4e388b8ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T13:55:11.917 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.913+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.917+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.917+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.928 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.924+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:11.928 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.924+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.940+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T13:55:11.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.940+0000 7f4e388b8ec0 20 zonegroup default 2019-12-08T13:55:11.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.940+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.948 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.945+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.945+0000 7f4e388b8ec0 10 Cannot find current period zone using local zone 2019-12-08T13:55:11.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.945+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.952 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.949+0000 7f4e388b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T13:55:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.949+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.962+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T13:55:11.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.962+0000 7f4e388b8ec0 20 rados->read ofs=0 len=0 2019-12-08T13:55:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.970+0000 7f4e388b8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T13:55:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.970+0000 7f4e388b8ec0 20 zone default found 2019-12-08T13:55:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.970+0000 7f4e388b8ec0 20 started sync module instance, tier type = 2019-12-08T13:55:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:11.970+0000 7f4e388b8ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T13:55:12.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=0 2019-12-08T13:55:12.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=1 2019-12-08T13:55:12.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=2 2019-12-08T13:55:12.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=3 2019-12-08T13:55:12.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=4 2019-12-08T13:55:12.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=5 2019-12-08T13:55:12.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.358+0000 7f4e388b8ec0 20 add_watcher() i=6 2019-12-08T13:55:12.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.392+0000 7f4e388b8ec0 20 add_watcher() i=7 2019-12-08T13:55:12.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.392+0000 7f4e388b8ec0 2 all 8 watchers are set, enabling cache 2019-12-08T13:55:12.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.392+0000 7f4df67dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T13:55:12.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.399+0000 7f4e25ae2700 0 -- 172.21.15.129:0/1934953687 >> v2:172.21.15.129:3300/0 conn(0x5587c770b300 msgr2=0x5587c76db2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T13:55:12.604 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.601+0000 7f4e25ae2700 0 --2- 172.21.15.129:0/1934953687 >> v2:172.21.15.129:3300/0 conn(0x5587c770b300 0x5587c76db2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-08T13:55:12.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:12.651+0000 7f4e1ffff700 0 -- 172.21.15.129:0/1318792293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587c7711680 msgr2=0x5587c76d4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T13:55:14.208 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.204+0000 7f4d6cff9700 20 reqs_thread_entry: start 2019-12-08T13:55:14.208 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.204+0000 7f4d667fc700 20 reqs_thread_entry: start 2019-12-08T13:55:14.208 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.204+0000 7f4e388b8ec0 20 init_complete bucket index max shards: 0 2019-12-08T13:55:14.209 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.205+0000 7f4d64ff9700 20 reqs_thread_entry: start 2019-12-08T13:55:14.209 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.205+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T13:55:14.266 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.262+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.283+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:14.300 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.297+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.353+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T13:55:14.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.414+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.441 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.437+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:14.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.450+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.488+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T13:55:14.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.538+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.559+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:14.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.571+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.619 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.615+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T13:55:14.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.657+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.680 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.673+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:14.685 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.682+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.716+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T13:55:14.756 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.753+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.772 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.768+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:14.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.777+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.811+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T13:55:14.885 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.882+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.912 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.909+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:14.925 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.922+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:14.979 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:14.976+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T13:55:15.054 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.051+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.081 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.077+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:15.100 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.097+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.155+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T13:55:15.225 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.222+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.243+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:15.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.254+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.318 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.315+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T13:55:15.398 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.395+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.424+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:15.447 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.443+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.497+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T13:55:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.545+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.565+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:15.579 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.576+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.613 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.609+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T13:55:15.659 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.655+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.672+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:15.685 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.681+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.712+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T13:55:15.798 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.795+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.835 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.831+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:15.852 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.848+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.906 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.903+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T13:55:15.972 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.969+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:15.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:15.994+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.008 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.005+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.066 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.062+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T13:55:16.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.140+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.168+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.182+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.228+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T13:55:16.313 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.309+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.341+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.361 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.358+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.425 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.421+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T13:55:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.503+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.532 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.528+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.537+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.588+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T13:55:16.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.642+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.660+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.675+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.759 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.755+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T13:55:16.816 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.812+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.838 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.834+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.846+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.893 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.889+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T13:55:16.936 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.932+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.949+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:16.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:16.958+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.008 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.005+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T13:55:17.075 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.071+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.098 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.095+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.112 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.108+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.155+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T13:55:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.204+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.223+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.234 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.231+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.269+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T13:55:17.360 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.357+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.395 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.392+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.407+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.460+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T13:55:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.535+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.563 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.560+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.575 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.571+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.637 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.633+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T13:55:17.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.696+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.716+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.730+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.773+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T13:55:17.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.823+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.843 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.839+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.846+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.885 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.881+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T13:55:17.940 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.936+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:17.955 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.952+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:17.963 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:17.960+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.005 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.001+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T13:55:18.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.041+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.061 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.058+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:18.071 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.068+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.099 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.095+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T13:55:18.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.190+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.227+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:18.243 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.239+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.300 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.296+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T13:55:18.373 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.370+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.406 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.403+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:18.425 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.421+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.479 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.475+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T13:55:18.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.538+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.565 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.562+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.578+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.619 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.615+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T13:55:18.684 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.680+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.708+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:18.727 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.723+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.764 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.760+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T13:55:18.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.805+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.823 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.820+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T13:55:18.832 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.828+0000 7f4e388b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T13:55:18.942 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.939+0000 7f4e388b8ec0 20 remove_watcher() i=0 2019-12-08T13:55:18.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:18.939+0000 7f4e388b8ec0 2 removed watcher, disabling cache 2019-12-08T13:55:19.004 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.000+0000 7f4e388b8ec0 20 remove_watcher() i=1 2019-12-08T13:55:19.063 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.060+0000 7f4e388b8ec0 20 remove_watcher() i=2 2019-12-08T13:55:19.136 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.133+0000 7f4e388b8ec0 20 remove_watcher() i=3 2019-12-08T13:55:19.183 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.179+0000 7f4e388b8ec0 20 remove_watcher() i=4 2019-12-08T13:55:19.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.222+0000 7f4e388b8ec0 20 remove_watcher() i=5 2019-12-08T13:55:19.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.284+0000 7f4e388b8ec0 20 remove_watcher() i=6 2019-12-08T13:55:19.354 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T13:55:19.350+0000 7f4e388b8ec0 20 remove_watcher() i=7 2019-12-08T13:55:19.366 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-12-08T13:55:19.885 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :clean 2019-12-08T13:55:20.483 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :compileJava 2019-12-08T13:55:20.483 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :processResources 2019-12-08T13:55:20.483 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :classes 2019-12-08T13:55:20.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:20.684 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :compileTestJava 2019-12-08T13:55:20.684 INFO:teuthology.orchestra.run.smithi129.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-08T13:55:20.684 INFO:teuthology.orchestra.run.smithi129.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-08T13:55:20.684 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:20.684 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :processTestResources NO-SOURCE 2019-12-08T13:55:20.684 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :testClasses 2019-12-08T13:55:23.282 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:23.282 INFO:teuthology.orchestra.run.smithi129.stdout:> Task :test 2019-12-08T13:55:23.282 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:23.282 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-12-08T13:55:27.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:27.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-12-08T13:55:35.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:35.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-12-08T13:55:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:37.882 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-12-08T13:55:43.687 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:43.688 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-12-08T13:55:44.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:44.982 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-12-08T13:55:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:55:51.982 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-12-08T13:56:46.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:56:46.982 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-12-08T13:56:50.682 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:56:50.682 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-12-08T13:56:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:56:53.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-12-08T13:56:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:56:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-12-08T13:57:02.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:02.582 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-12-08T13:57:10.282 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:10.282 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-12-08T13:57:10.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:10.482 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-12-08T13:57:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:11.582 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-12-08T13:57:14.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:14.482 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-12-08T13:57:14.720 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:13:46.147 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.728 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:13:46.154 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.736 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:13:46.160 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.740 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:13:46.165 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.748 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:13:46.335 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.753 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:13:46.340 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.906 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:13:46.332 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:14.911 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:13:46.336 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:15.028 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:13:46.615 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:57:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:18.582 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-12-08T13:57:57.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:57:57.182 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-12-08T13:58:07.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:07.782 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-12-08T13:58:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:11.182 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-12-08T13:58:20.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:20.082 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-12-08T13:58:23.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:23.182 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-12-08T13:58:30.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:30.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-12-08T13:58:32.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:32.582 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-12-08T13:58:38.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:38.082 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-12-08T13:58:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:39.482 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-12-08T13:58:45.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:45.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-12-08T13:58:48.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:48.082 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-12-08T13:58:54.682 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:54.682 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-12-08T13:58:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:58:57.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-12-08T13:59:04.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:59:04.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-12-08T13:59:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:59:09.182 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-12-08T13:59:13.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T13:59:13.882 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-12-08T13:59:31.398 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:16:02.824 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.451 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:16:02.877 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.452 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:16:02.876 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.460 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:16:02.884 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.475 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:16:03.064 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.480 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:16:03.070 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.531 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:16:02.957 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.548 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:16:02.972 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T13:59:31.570 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:16:03.159 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:00:02.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:00:02.082 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-12-08T14:00:17.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:00:17.482 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-12-08T14:02:24.732 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:18:56.158 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.741 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:18:56.167 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.747 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:18:56.171 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.751 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:18:56.176 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.752 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:18:56.342 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.758 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:18:56.348 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.847 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:18:56.272 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.886 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:18:56.477 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:02:24.913 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:18:56.337 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:05:20.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:05:20.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-12-08T14:05:34.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:05:34.782 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-12-08T14:05:36.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:05:36.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-12-08T14:05:42.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:05:42.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-12-08T14:05:46.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:05:46.882 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-12-08T14:05:55.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:05:55.382 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-12-08T14:06:32.682 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:06:32.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-12-08T14:06:39.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:06:39.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-12-08T14:06:39.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:06:39.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-12-08T14:06:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:06:40.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-12-08T14:07:28.658 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T14:07:28.555+0000 f6fe5700 -1 handle_data failed with exit code -104 2019-12-08T14:07:30.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:07:30.383 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-12-08T14:07:37.741 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:24:09.167 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.749 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:24:09.173 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.759 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:24:09.183 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.764 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:24:09.355 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.765 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:24:09.189 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.768 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:24:09.358 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:07:37.782 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-12-08T14:07:37.937 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:24:09.363 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.988 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:24:09.412 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:37.991 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:24:09.581 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.826 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:24:24.252 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.856 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:24:24.280 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.860 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:24:24.286 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.867 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:24:24.291 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.881 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:24:24.307 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.886 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:24:24.476 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.887 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:24:24.311 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.905 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:24:24.492 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:07:52.927 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:24:24.517 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:09:03.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:09:03.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-12-08T14:09:16.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:09:16.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-12-08T14:10:11.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:10:11.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-12-08T14:10:19.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:10:19.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-12-08T14:12:47.114 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:29:18.540 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.122 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:29:18.547 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.130 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:29:18.554 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.135 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:29:18.725 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.135 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:29:18.559 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.139 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:29:18.730 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.289 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:29:18.714 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.304 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:29:18.727 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:12:47.335 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:29:18.925 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.188 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:32:45.613 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.202 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:32:45.625 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.218 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:32:45.643 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.224 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:32:45.647 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.254 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:32:45.845 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.256 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:32:45.847 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.326 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:32:45.751 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.346 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:32:45.769 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:14.383 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:32:45.974 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:16:58.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:16:58.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-12-08T14:17:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:17:04.684 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-12-08T14:17:57.250 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:34:28.675 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.256 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:34:28.681 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.260 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:34:28.684 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.264 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:34:28.687 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.276 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:34:28.867 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.290 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:34:28.881 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.335 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:34:28.758 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.349 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:34:28.774 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:17:57.408 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:34:28.999 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:18:36.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:18:36.684 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-12-08T14:18:44.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:18:44.184 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-12-08T14:19:41.084 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:19:41.084 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-12-08T14:19:48.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:19:48.884 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-12-08T14:20:03.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:03.584 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-12-08T14:20:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:18.283 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-12-08T14:20:18.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:18.884 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-12-08T14:20:19.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:19.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-12-08T14:20:21.083 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:21.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-12-08T14:20:32.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:32.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-12-08T14:20:39.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:39.584 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-12-08T14:20:56.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:20:56.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-12-08T14:21:02.083 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:21:02.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-12-08T14:21:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:21:15.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-12-08T14:21:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:21:25.784 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-12-08T14:21:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:21:48.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-12-08T14:21:55.083 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:21:55.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-12-08T14:22:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:22:27.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-12-08T14:22:40.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:22:40.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-12-08T14:23:11.404 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:39:42.829 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.414 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:39:42.840 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.420 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:39:42.843 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.423 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:39:42.846 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.425 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:39:43.016 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.434 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:39:43.020 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.613 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:39:43.036 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.622 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:39:43.213 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:11.658 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:39:43.083 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:23:32.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:23:32.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-12-08T14:23:39.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:23:39.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-12-08T14:24:40.540 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:41:11.965 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.548 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:41:11.973 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.589 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:41:12.012 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.591 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:41:12.014 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.610 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:41:12.201 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.612 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:41:12.204 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.645 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:41:12.068 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.646 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:41:12.071 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:40.707 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:41:12.298 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:24:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:24:52.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-12-08T14:25:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:25:48.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-12-08T14:26:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:26:43.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-12-08T14:26:56.090 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:43:27.515 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.133 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:43:27.553 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.180 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:43:27.603 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.194 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:43:27.617 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.204 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:43:27.795 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.208 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:43:27.799 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.234 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:43:27.659 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.289 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:43:27.713 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:26:56.301 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:43:27.892 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:27:43.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:27:43.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-12-08T14:28:21.472 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:44:52.897 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.476 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:44:52.902 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.487 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:44:52.910 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.491 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:44:52.914 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.503 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:44:53.093 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.517 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:44:53.108 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.546 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:44:52.972 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.556 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:44:52.979 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:28:21.693 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:44:53.284 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:29:37.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:29:37.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-12-08T14:31:13.984 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:31:13.984 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-12-08T14:33:02.224 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:49:33.649 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.249 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:49:33.675 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.289 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:49:33.713 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.299 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:49:33.722 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.309 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:49:33.899 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.318 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:49:33.902 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.433 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:49:33.856 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.436 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:49:33.861 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:02.526 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:49:34.115 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.397 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:50:03.821 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.404 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:50:03.829 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.408 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:50:03.832 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.411 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:50:04.001 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.412 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:50:03.836 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.416 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:50:04.006 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.444 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:50:03.869 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.453 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:50:03.877 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:32.468 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:50:04.057 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:33:46.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:33:46.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-12-08T14:35:50.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:35:50.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-12-08T14:38:45.204 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:55:16.627 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.209 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:55:16.635 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.220 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:55:16.644 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.223 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:55:16.647 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.228 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:55:16.816 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.231 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:55:16.820 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.406 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:55:16.829 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.448 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:55:16.873 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:45.560 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:55:17.146 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:38:48.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:38:48.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-12-08T14:41:28.522 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:57:59.947 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.566 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:57:59.991 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.571 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:57:59.994 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.604 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:58:00.027 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.628 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:58:00.217 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.633 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:58:00.222 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.638 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:01:58:00.063 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.676 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:01:58:00.099 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:28.736 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:01:58:00.325 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:41:31.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:41:31.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-12-08T14:43:55.138 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:00:26.563 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.142 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:00:26.568 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.155 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:00:26.579 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.162 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:00:26.585 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.169 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:00:26.757 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.176 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:00:26.765 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.240 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:00:26.665 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.277 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:00:26.700 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:43:55.328 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:00:26.917 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:44:53.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:44:53.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-12-08T14:46:55.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:46:55.884 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-12-08T14:49:09.916 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:05:41.341 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:09.924 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:05:41.349 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:09.932 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:05:41.355 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:09.938 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:05:41.361 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:09.938 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:05:41.528 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:09.943 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:05:41.533 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:10.121 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:05:41.544 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:10.123 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:05:41.713 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:10.136 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:05:41.561 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:54.981 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:06:26.405 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.002 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:06:26.425 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.020 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:06:26.445 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.023 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:06:26.446 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.060 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:06:26.483 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.064 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:06:26.489 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.070 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:06:26.660 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.075 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:06:26.665 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:49:55.211 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:06:26.801 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:50:27.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:50:27.984 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-12-08T14:51:28.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:51:28.784 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-12-08T14:53:26.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:53:26.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-12-08T14:54:22.257 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:10:53.682 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.263 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:10:53.688 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.266 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:10:53.690 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.272 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:10:53.696 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.277 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:10:53.867 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.312 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:10:53.902 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.433 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:10:53.858 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.470 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:10:53.893 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:22.513 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:10:54.103 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:54:47.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:54:47.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-12-08T14:56:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:56:14.383 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-12-08T14:56:59.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:56:59.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-12-08T14:57:48.283 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:57:48.283 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-12-08T14:57:56.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:57:56.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-12-08T14:58:09.283 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:58:09.283 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-12-08T14:58:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:58:18.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-12-08T14:58:21.420 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:14:52.845 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.445 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:14:52.871 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.463 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:14:52.887 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.469 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:14:52.893 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.505 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:14:53.096 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.508 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:14:53.099 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.542 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:14:52.967 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.567 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:14:52.990 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:21.673 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:14:53.264 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:58:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:58:31.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-12-08T14:58:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:58:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-12-08T14:58:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:58:49.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-12-08T14:58:57.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:58:57.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-12-08T14:59:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:59:09.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-12-08T14:59:18.383 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:59:18.383 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-12-08T14:59:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:59:29.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-12-08T14:59:34.421 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:16:05.846 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.429 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:16:05.855 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.440 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:16:05.864 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.444 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:16:05.868 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.453 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:16:06.044 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.491 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:16:06.083 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.500 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:16:05.924 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.520 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:16:05.945 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:34.604 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:16:06.195 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T14:59:36.083 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:59:36.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-12-08T14:59:48.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:59:48.184 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-12-08T14:59:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T14:59:56.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-12-08T15:00:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:00:08.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-12-08T15:00:16.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:00:16.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-12-08T15:00:30.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:00:30.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-12-08T15:00:36.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:00:36.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-12-08T15:00:49.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:00:49.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-12-08T15:00:56.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:00:56.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-12-08T15:01:05.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:01:05.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-12-08T15:01:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:01:11.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-12-08T15:01:23.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:01:23.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-12-08T15:01:31.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:01:31.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-12-08T15:01:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:01:43.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-12-08T15:01:49.083 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:01:49.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-12-08T15:02:00.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:00.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-12-08T15:02:03.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:03.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-12-08T15:02:09.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:09.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-12-08T15:02:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:13.083 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-12-08T15:02:21.182 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:21.183 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-12-08T15:02:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:24.283 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-12-08T15:02:31.682 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:31.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-12-08T15:02:34.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:34.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-12-08T15:02:42.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:42.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-12-08T15:02:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:52.283 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-12-08T15:02:57.282 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:57.283 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-12-08T15:02:57.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:57.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-12-08T15:02:58.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:02:58.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-12-08T15:03:02.382 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:02.383 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-12-08T15:03:10.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:10.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-12-08T15:03:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:14.783 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-12-08T15:03:22.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:22.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-12-08T15:03:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:25.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-12-08T15:03:32.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:32.583 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-12-08T15:03:35.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:35.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-12-08T15:03:42.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:42.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-12-08T15:03:46.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:46.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-12-08T15:03:56.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:03:56.483 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-12-08T15:04:34.882 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:04:34.883 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-12-08T15:04:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:04:42.683 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-12-08T15:04:45.641 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:21:17.066 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.647 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:21:17.073 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.656 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:21:17.080 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.660 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:21:17.084 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.663 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:21:17.254 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.668 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:21:17.259 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.836 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:21:17.262 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.839 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:21:17.263 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:04:45.914 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:21:17.506 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:05:22.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:05:22.983 INFO:teuthology.orchestra.run.smithi129.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-12-08T15:05:35.062 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:05:35.062 INFO:teuthology.orchestra.run.smithi129.stdout:---------------------------------------------------------------------- 2019-12-08T15:05:35.062 INFO:teuthology.orchestra.run.smithi129.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-12-08T15:05:35.062 INFO:teuthology.orchestra.run.smithi129.stdout:---------------------------------------------------------------------- 2019-12-08T15:05:35.063 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:05:35.063 INFO:teuthology.orchestra.run.smithi129.stdout:BUILD SUCCESSFUL in 1h 10m 15s 2019-12-08T15:05:35.063 INFO:teuthology.orchestra.run.smithi129.stdout:5 actionable tasks: 5 executed 2019-12-08T15:05:35.067 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T15:05:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.202+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.228 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.223+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.228 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.223+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.233+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.238 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.233+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.242 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.238+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.242 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.238+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.250 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.245+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:35.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.453+0000 7fd82b325ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:05:35.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.453+0000 7fd82b325ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:05:35.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.453+0000 7fd82b325ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:05:35.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.453+0000 7fd82b325ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:05:35.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.453+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.464+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:35.469 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.464+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.482 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.478+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:05:35.483 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.478+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.489+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:35.494 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.489+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.505+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:05:35.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.505+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.509+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.510+0000 7fd82b325ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:05:35.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.510+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.519 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.515+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.519 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.515+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.527 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.523+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:35.528 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.523+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.538+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:05:35.542 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.538+0000 7fd82b325ec0 20 zonegroup default 2019-12-08T15:05:35.543 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.538+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.544+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.544+0000 7fd82b325ec0 10 Cannot find current period zone using local zone 2019-12-08T15:05:35.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.544+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.549+0000 7fd82b325ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:35.553 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.549+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.556+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:35.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.556+0000 7fd82b325ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:35.569 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.564+0000 7fd82b325ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:05:35.569 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.564+0000 7fd82b325ec0 20 zone default found 2019-12-08T15:05:35.570 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.564+0000 7fd82b325ec0 20 started sync module instance, tier type = 2019-12-08T15:05:35.570 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.564+0000 7fd82b325ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:05:35.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.939+0000 7fd82b325ec0 20 add_watcher() i=0 2019-12-08T15:05:35.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.940+0000 7fd82b325ec0 20 add_watcher() i=1 2019-12-08T15:05:35.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.940+0000 7fd82b325ec0 20 add_watcher() i=2 2019-12-08T15:05:35.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.940+0000 7fd82b325ec0 20 add_watcher() i=3 2019-12-08T15:05:35.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.940+0000 7fd82b325ec0 20 add_watcher() i=4 2019-12-08T15:05:35.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.940+0000 7fd82b325ec0 20 add_watcher() i=5 2019-12-08T15:05:35.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.940+0000 7fd82b325ec0 20 add_watcher() i=6 2019-12-08T15:05:35.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.980+0000 7fd82b325ec0 20 add_watcher() i=7 2019-12-08T15:05:35.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.980+0000 7fd82b325ec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:05:35.985 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.980+0000 7fd7e8fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:05:35.990 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:35.986+0000 7fd81854f700 0 -- 172.21.15.129:0/2679481969 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x564190f50690 msgr2=0x564190f50aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T15:05:36.192 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:36.188+0000 7fd81854f700 0 --2- 172.21.15.129:0/2679481969 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x564190f50690 0x564190f50aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-08T15:05:36.243 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:36.238+0000 7fd81854f700 0 -- 172.21.15.129:0/360663880 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7fd75c0224b0 msgr2=0x7fd75c0249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:05:37.566 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.562+0000 7fd75affd700 20 reqs_thread_entry: start 2019-12-08T15:05:37.567 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.562+0000 7fd758ff9700 20 reqs_thread_entry: start 2019-12-08T15:05:37.567 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.562+0000 7fd82b325ec0 20 init_complete bucket index max shards: 0 2019-12-08T15:05:37.567 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.563+0000 7fd746ffd700 20 reqs_thread_entry: start 2019-12-08T15:05:37.568 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.563+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T15:05:37.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.623+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:37.645 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.641+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:37.654 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.650+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:37.689 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.682+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T15:05:37.724 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.720+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:37.740 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.736+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:37.752 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:37.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1164', time=2019-12-08T16:05:31.409325+0000, chain.objs.size()=5 2019-12-08T15:05:37.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.66__shadow_.Sv0c3GTOyeHumScMEClacPG5Bayhu28_1 2019-12-08T15:05:37.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.66__shadow_.Sv0c3GTOyeHumScMEClacPG5Bayhu28_2 2019-12-08T15:05:37.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.66__shadow_.Sv0c3GTOyeHumScMEClacPG5Bayhu28_3 2019-12-08T15:05:37.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.66__shadow_.Sv0c3GTOyeHumScMEClacPG5Bayhu28_4 2019-12-08T15:05:37.753 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.748+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.66__shadow_.Sv0c3GTOyeHumScMEClacPG5Bayhu28_5 2019-12-08T15:05:37.938 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:37.934+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:38.019 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.015+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T15:05:38.097 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.093+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:38.114 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.110+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1172', time=2019-12-08T16:02:42.305631+0000, chain.objs.size()=9 2019-12-08T15:05:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.1 2019-12-08T15:05:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.1_1 2019-12-08T15:05:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.2 2019-12-08T15:05:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.2_1 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.3 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.3_1 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.4 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.4_1 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~ZxhjotIFjgsJW0EuIVP-Y6Td3CSxOhm.5 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf', time=2019-12-08T16:10:15.589692+0000, chain.objs.size()=9 2019-12-08T15:05:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__multipart_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.7 2019-12-08T15:05:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__shadow_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.7_1 2019-12-08T15:05:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.124+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__multipart_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.8 2019-12-08T15:05:38.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.383+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__shadow_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.8_1 2019-12-08T15:05:38.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.383+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__multipart_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.9 2019-12-08T15:05:38.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.383+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__shadow_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.9_1 2019-12-08T15:05:38.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.544+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__multipart_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.10 2019-12-08T15:05:38.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.544+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__shadow_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.10_1 2019-12-08T15:05:38.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.545+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.74__multipart_key1.2~D62bMqKBwS7TwyxU2iYWPTc_HAh13uf.11 2019-12-08T15:05:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.873+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:38.987 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:38.982+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T15:05:39.025 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.021+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:39.048 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.044+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.052+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:39.091 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.086+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T15:05:39.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.149+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.174+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:39.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.190+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:39.250 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.245+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T15:05:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.307+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.328+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.344+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.344+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1149', time=2019-12-08T16:00:10.827862+0000, chain.objs.size()=9 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.344+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.1 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.1_1 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.2 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.2_1 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.3 2019-12-08T15:05:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.3_1 2019-12-08T15:05:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.4 2019-12-08T15:05:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.4_1 2019-12-08T15:05:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.345+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.5 2019-12-08T15:05:39.606 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.601+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:39.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.696+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T15:05:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.742+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:39.761 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.757+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:39.790 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-08T15:05:39.790 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1318', time=2019-12-08T16:12:50.106591+0000, chain.objs.size()=9 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.1 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.1_1 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.2 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.2_1 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.3 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.3_1 2019-12-08T15:05:39.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.4 2019-12-08T15:05:39.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.4_1 2019-12-08T15:05:39.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~Rdimx5cV5sfBMnT-Jj0qRJ6Ec056c3y.5 2019-12-08T15:05:39.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1377', time=2019-12-08T16:19:43.084686+0000, chain.objs.size()=9 2019-12-08T15:05:39.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__multipart_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.1 2019-12-08T15:05:39.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__shadow_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.1_1 2019-12-08T15:05:39.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:39.785+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__multipart_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.2 2019-12-08T15:05:40.120 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.115+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__shadow_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.2_1 2019-12-08T15:05:40.142 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.137+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__multipart_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.3 2019-12-08T15:05:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.137+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__shadow_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.3_1 2019-12-08T15:05:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__multipart_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.4 2019-12-08T15:05:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__shadow_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.4_1 2019-12-08T15:05:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.77__multipart_key1.2~9Fdev3uCjmRXAJtjUIEjRNsk_iSziPx.5 2019-12-08T15:05:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~b3geZ-htcrgORLBv1c8R0IRAJukbliV', time=2019-12-08T16:20:14.952576+0000, chain.objs.size()=27 2019-12-08T15:05:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.1 2019-12-08T15:05:40.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.2 2019-12-08T15:05:40.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.3 2019-12-08T15:05:40.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.4 2019-12-08T15:05:40.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.138+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.5 2019-12-08T15:05:40.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.453+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.6 2019-12-08T15:05:40.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.453+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.7 2019-12-08T15:05:40.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.453+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.8 2019-12-08T15:05:40.458 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.453+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.9 2019-12-08T15:05:40.496 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.491+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.10 2019-12-08T15:05:40.496 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.491+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.11 2019-12-08T15:05:40.496 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.491+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.12 2019-12-08T15:05:40.496 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.491+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.13 2019-12-08T15:05:40.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.499+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.YILSs2NIiZL-reIBkamKhaQThRGwsuw.14 2019-12-08T15:05:40.503 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.499+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.8AZhcR85vALqWvGABccT21kONu9nLOj.15 2019-12-08T15:05:40.515 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.511+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.Cfn_iIDoomt9ducX7YaqLcfwDF5seXU.16 2019-12-08T15:05:40.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.634+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.czCBKfHODeitTiWAgJz0XdDa4FhReb3.17 2019-12-08T15:05:40.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.634+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.MKXNuqA5JiRjaJ6-xIpVLPCjtieKJ8Q.18 2019-12-08T15:05:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.703+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.tQhS9pof7xtR1Qus4nj8B8ZokBmHmIS.19 2019-12-08T15:05:40.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.779+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.APjTq0N9LX1BhquNePy_2HKUcSYAeZq.20 2019-12-08T15:05:40.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.810+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.SywRD6xI823lgTBf4OKm_lhK7atH1G6.21 2019-12-08T15:05:40.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.811+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.22 2019-12-08T15:05:40.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.811+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.23 2019-12-08T15:05:40.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.811+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.24 2019-12-08T15:05:40.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.811+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.25 2019-12-08T15:05:40.816 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.811+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.26 2019-12-08T15:05:40.816 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:40.811+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.78__multipart_key1.2~b3geZ-htcrgORLBv1c8R0IRAJukbliV.27 2019-12-08T15:05:41.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.046+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:41.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.200+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T15:05:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.262+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:41.291 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.286+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.298+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:41.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.335+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T15:05:41.381 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.376+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:41.396 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.392+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1180', time=2019-12-08T16:03:36.772380+0000, chain.objs.size()=9 2019-12-08T15:05:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.1 2019-12-08T15:05:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.1_1 2019-12-08T15:05:41.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.2 2019-12-08T15:05:41.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.2_1 2019-12-08T15:05:41.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.3 2019-12-08T15:05:41.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.3_1 2019-12-08T15:05:41.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.4 2019-12-08T15:05:41.411 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.4_1 2019-12-08T15:05:41.412 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.405+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~pS1hb54skbyz5hfXWcPM41Ekf9jW__5.5 2019-12-08T15:05:41.727 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.721+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.832+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T15:05:41.909 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.904+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:41.938 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.934+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:41.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:41.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1194', time=2019-12-08T16:04:32.495395+0000, chain.objs.size()=7 2019-12-08T15:05:41.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.1 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.1_1 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.1_2 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.2 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.2_1 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.2_2 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~7l0DE9_lRX2fqJgr4wKwO8lPkyLGWrt.3 2019-12-08T15:05:41.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1243', time=2019-12-08T16:06:34.868128+0000, chain.objs.size()=9 2019-12-08T15:05:41.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__multipart_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.1 2019-12-08T15:05:41.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__shadow_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.1_1 2019-12-08T15:05:41.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__multipart_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.2 2019-12-08T15:05:41.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__shadow_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.2_1 2019-12-08T15:05:41.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:41.959+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__multipart_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.3 2019-12-08T15:05:42.201 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.193+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__shadow_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.3_1 2019-12-08T15:05:42.201 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.193+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__multipart_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.4 2019-12-08T15:05:42.201 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.193+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__shadow_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.4_1 2019-12-08T15:05:42.348 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.344+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.71__multipart_key1.2~7tmFhbkJYlA5g-YnKU8OxySZDcQiXuM.5 2019-12-08T15:05:42.574 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.570+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.662+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T15:05:42.710 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.706+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:42.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.723+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:42.735 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.731+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.788+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T15:05:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.831+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:42.852 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.848+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:42.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:42.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1206', time=2019-12-08T16:05:23.431612+0000, chain.objs.size()=7 2019-12-08T15:05:42.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.1 2019-12-08T15:05:42.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.1_1 2019-12-08T15:05:42.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.1_2 2019-12-08T15:05:42.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.2 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.2_1 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.2_2 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~sEmKJUWJrYZRyW6xH_FG1WxPs9zhmB1.3 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.2188', time=2019-12-08T17:05:26.672171+0000, chain.objs.size()=9 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__multipart_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.1 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__shadow_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.1_1 2019-12-08T15:05:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__multipart_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.2 2019-12-08T15:05:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__shadow_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.2_1 2019-12-08T15:05:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:42.867+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__multipart_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.3 2019-12-08T15:05:43.047 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.042+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__shadow_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.3_1 2019-12-08T15:05:43.062 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.057+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__multipart_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.4 2019-12-08T15:05:43.153 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.149+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__shadow_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.4_1 2019-12-08T15:05:43.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.149+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.119__multipart_key1/data/file.mpg.2~PvKVr8P-RNeGz2Y0bT3IBc-bhT-B9mv.5 2019-12-08T15:05:43.330 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.326+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.484+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T15:05:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.544+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:43.598 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.593+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:43.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.608+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.668+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T15:05:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.728+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:43.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.750+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:43.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:43.781 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1287', time=2019-12-08T16:09:06.443585+0000, chain.objs.size()=9 2019-12-08T15:05:43.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__multipart_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.1 2019-12-08T15:05:43.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__shadow_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.1_1 2019-12-08T15:05:43.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__multipart_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.2 2019-12-08T15:05:43.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__shadow_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.2_1 2019-12-08T15:05:43.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__multipart_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.3 2019-12-08T15:05:43.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__shadow_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.3_1 2019-12-08T15:05:43.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__multipart_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.4 2019-12-08T15:05:43.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__shadow_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.4_1 2019-12-08T15:05:43.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:43.774+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.73__multipart_file.mpg.2~swJkd6pGLvCEbtElymtOwYIAbovWQAu.5 2019-12-08T15:05:44.080 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.075+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:44.183 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.176+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T15:05:44.221 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.217+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:44.239 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.234+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:44.248 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.244+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:44.292 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.283+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T15:05:44.347 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.343+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:44.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.363+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:44.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.380+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:44.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.380+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.2172', time=2019-12-08T17:04:37.615107+0000, chain.objs.size()=9 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.380+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__multipart_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.1 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.380+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__shadow_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.1_1 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.380+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__multipart_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.2 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.380+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__shadow_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.2_1 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.381+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__multipart_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.3 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.381+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__shadow_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.3_1 2019-12-08T15:05:44.389 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.381+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__multipart_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.4 2019-12-08T15:05:44.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.381+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__shadow_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.4_1 2019-12-08T15:05:44.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.381+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.118__multipart_key1.2~pUA7JWdmH2-cl6zec5O4RGNYadHfp8w.5 2019-12-08T15:05:44.758 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.753+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:44.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.843+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T15:05:44.891 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.884+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:44.904 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.899+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:44.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:44.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1151', time=2019-12-08T16:00:04.180107+0000, chain.objs.size()=9 2019-12-08T15:05:44.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.1 2019-12-08T15:05:44.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.1_1 2019-12-08T15:05:44.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.2 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.2_1 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.3 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.3_1 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.4 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__shadow_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.4_1 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.64__multipart_key1.2~-hxkKYmTFwM3WGvYMMN8Z_bTameU9g9.5 2019-12-08T15:05:44.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1200', time=2019-12-08T16:05:19.396275+0000, chain.objs.size()=7 2019-12-08T15:05:44.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.1 2019-12-08T15:05:44.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.1_1 2019-12-08T15:05:44.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:44.915+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.1_2 2019-12-08T15:05:45.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.253+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.2 2019-12-08T15:05:45.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.253+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.2_1 2019-12-08T15:05:45.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.253+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.2_2 2019-12-08T15:05:45.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.253+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~PpAUvT8pJHcls39pMoDRFBFIuMeXuss.3 2019-12-08T15:05:45.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.567+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:45.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.653+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T15:05:45.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.691+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:45.710 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.706+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:45.718 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.714+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:45.752 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.747+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T15:05:45.803 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.799+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:45.820 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.816+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct', time=2019-12-08T15:56:46.184365+0000, chain.objs.size()=9 2019-12-08T15:05:45.838 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__multipart_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.1 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__shadow_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.1_1 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__multipart_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.2 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__shadow_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.2_1 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__multipart_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.3 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__shadow_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.3_1 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__multipart_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.4 2019-12-08T15:05:45.839 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__shadow_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.4_1 2019-12-08T15:05:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.52__multipart_key1.2~2CA0sqjttQCOxhhJMIQNBgHBCsCKYct.5 2019-12-08T15:05:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1332', time=2019-12-08T16:15:00.423203+0000, chain.objs.size()=9 2019-12-08T15:05:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.1 2019-12-08T15:05:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.1_1 2019-12-08T15:05:45.840 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:45.833+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.2 2019-12-08T15:05:46.124 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.119+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.2_1 2019-12-08T15:05:46.124 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.119+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.3 2019-12-08T15:05:46.124 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.119+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.3_1 2019-12-08T15:05:46.124 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.119+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.4 2019-12-08T15:05:46.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.160+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.4_1 2019-12-08T15:05:46.164 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.160+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~p5ryyPHZ9V22wFOCVKy7Oa8mjuxEkNF.5 2019-12-08T15:05:46.586 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.582+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:46.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.697+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T15:05:46.750 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.746+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:46.768 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.763+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:46.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:46.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1077', time=2019-12-08T15:58:01.097223+0000, chain.objs.size()=2 2019-12-08T15:05:46.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.56__shadow_.-bLcmgdLImK4w3fGuuYxlVAoXO0kows_1 2019-12-08T15:05:46.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.56__shadow_.-bLcmgdLImK4w3fGuuYxlVAoXO0kows_2 2019-12-08T15:05:46.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1342', time=2019-12-08T16:16:57.476158+0000, chain.objs.size()=7 2019-12-08T15:05:46.782 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.1 2019-12-08T15:05:46.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.1_1 2019-12-08T15:05:46.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.1_2 2019-12-08T15:05:46.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.2 2019-12-08T15:05:46.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.2_1 2019-12-08T15:05:46.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.2_2 2019-12-08T15:05:46.784 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:46.777+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~n6AqnRxD1AW-QxnT9bTW4XtKdZqqFH5.3 2019-12-08T15:05:47.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.161+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:47.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.251+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T15:05:47.318 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.314+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.338 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.333+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:47.346 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.342+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.397+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T15:05:47.482 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.478+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.508+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:47.526 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.522+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.569 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.564+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T15:05:47.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.628+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.659+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:47.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.672+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.714 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.710+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T15:05:47.760 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.756+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.773+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:47.787 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.783+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.814+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T15:05:47.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.857+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.878+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:47.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.891+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.928 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.924+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T15:05:47.980 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.975+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:47.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:47.993+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:48.011 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.006+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:48.054 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.050+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T15:05:48.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.103+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:48.143 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.139+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:48.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:48.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1362', time=2019-12-08T16:18:39.218159+0000, chain.objs.size()=9 2019-12-08T15:05:48.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__multipart_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.1 2019-12-08T15:05:48.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__shadow_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.1_1 2019-12-08T15:05:48.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__multipart_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.2 2019-12-08T15:05:48.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__shadow_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.2_1 2019-12-08T15:05:48.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__multipart_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.3 2019-12-08T15:05:48.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__shadow_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.3_1 2019-12-08T15:05:48.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__multipart_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.4 2019-12-08T15:05:48.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__shadow_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.4_1 2019-12-08T15:05:48.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.154+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.76__multipart_key1.2~1JMxkG6GTr8Wyg_ykAbRUB8OlXgKTQX.5 2019-12-08T15:05:48.506 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.502+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:48.625 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.619+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T15:05:48.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.672+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:48.693 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.689+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:48.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:48.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1262', time=2019-12-08T16:07:32.801801+0000, chain.objs.size()=9 2019-12-08T15:05:48.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__multipart_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.1 2019-12-08T15:05:48.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__shadow_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.1_1 2019-12-08T15:05:48.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__multipart_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.2 2019-12-08T15:05:48.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__shadow_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.2_1 2019-12-08T15:05:48.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__multipart_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.3 2019-12-08T15:05:48.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__shadow_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.3_1 2019-12-08T15:05:48.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__multipart_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.4 2019-12-08T15:05:48.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__shadow_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.4_1 2019-12-08T15:05:48.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.711+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.72__multipart_key1.2~lbjjBQpfknzQSGF_HSjCW1meJU0tcY_.5 2019-12-08T15:05:48.986 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:48.981+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:49.107 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.103+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T15:05:49.147 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.143+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:49.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.157+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:49.174 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:49.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1325', time=2019-12-08T16:13:50.219809+0000, chain.objs.size()=9 2019-12-08T15:05:49.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.1 2019-12-08T15:05:49.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.1_1 2019-12-08T15:05:49.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.2 2019-12-08T15:05:49.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.2_1 2019-12-08T15:05:49.175 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.3 2019-12-08T15:05:49.176 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.3_1 2019-12-08T15:05:49.176 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.4 2019-12-08T15:05:49.176 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.4_1 2019-12-08T15:05:49.176 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.170+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~FDMDviiMLzEODfDjGIKgPp5uPJ62xVz.5 2019-12-08T15:05:49.505 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.501+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:49.604 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.600+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T15:05:49.644 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.639+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:49.660 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.656+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:49.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-08T15:05:49.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1337', time=2019-12-08T16:16:10.068284+0000, chain.objs.size()=7 2019-12-08T15:05:49.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.1 2019-12-08T15:05:49.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.1_1 2019-12-08T15:05:49.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.1_2 2019-12-08T15:05:49.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.2 2019-12-08T15:05:49.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.2_1 2019-12-08T15:05:49.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.2_2 2019-12-08T15:05:49.674 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.668+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~8K44-XW_fQp07Hw9o7QpWovlJM8H6Mv.3 2019-12-08T15:05:49.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:49.910+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.003+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T15:05:50.052 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.048+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:50.067 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.062+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:50.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-08T15:05:50.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1188', time=2019-12-08T16:04:05.574894+0000, chain.objs.size()=9 2019-12-08T15:05:50.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.1 2019-12-08T15:05:50.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.1_1 2019-12-08T15:05:50.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.2 2019-12-08T15:05:50.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.2_1 2019-12-08T15:05:50.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.3 2019-12-08T15:05:50.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.3_1 2019-12-08T15:05:50.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.4 2019-12-08T15:05:50.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__shadow_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.4_1 2019-12-08T15:05:50.083 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.67__multipart_key1.2~GsMnGbGreACiQ73RDsFsIkTTKlnBm-t.5 2019-12-08T15:05:50.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 20 garbage collection: RGWGC::process iterating over entry tag='9161132f-da48-40b4-a853-f341a1971d48.4530.1347', time=2019-12-08T16:17:00.778654+0000, chain.objs.size()=7 2019-12-08T15:05:50.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.1 2019-12-08T15:05:50.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.1_1 2019-12-08T15:05:50.084 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.077+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.1_2 2019-12-08T15:05:50.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.404+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.2 2019-12-08T15:05:50.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.503+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.2_1 2019-12-08T15:05:50.508 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.503+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__shadow_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.2_2 2019-12-08T15:05:50.508 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.503+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9161132f-da48-40b4-a853-f341a1971d48.4507.75__multipart_key1.2~KzMBXi9HyXMv4ROXxOSFY-nvLqKFkeM.3 2019-12-08T15:05:50.681 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.676+0000 7fd82b325ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-08T15:05:50.776 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.772+0000 7fd82b325ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T15:05:50.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.810+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:50.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.826+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:50.838 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.834+0000 7fd82b325ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:50.919 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.914+0000 7fd82b325ec0 20 remove_watcher() i=0 2019-12-08T15:05:50.919 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.914+0000 7fd82b325ec0 2 removed watcher, disabling cache 2019-12-08T15:05:50.970 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:50.966+0000 7fd82b325ec0 20 remove_watcher() i=1 2019-12-08T15:05:51.006 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.002+0000 7fd82b325ec0 20 remove_watcher() i=2 2019-12-08T15:05:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.040+0000 7fd82b325ec0 20 remove_watcher() i=3 2019-12-08T15:05:51.075 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.071+0000 7fd82b325ec0 20 remove_watcher() i=4 2019-12-08T15:05:51.147 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.142+0000 7fd82b325ec0 20 remove_watcher() i=5 2019-12-08T15:05:51.185 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.181+0000 7fd82b325ec0 20 remove_watcher() i=6 2019-12-08T15:05:51.216 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.212+0000 7fd82b325ec0 20 remove_watcher() i=7 2019-12-08T15:05:51.224 INFO:teuthology.orchestra.run.smithi129:> radosgw-admin gc process --include-all 2019-12-08T15:05:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.329+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.355+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.360 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.355+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.368+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.372 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.368+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.375+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.375+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.388 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.383+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:51.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.555+0000 7fc3c7efaec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:05:51.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.555+0000 7fc3c7efaec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:05:51.560 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.555+0000 7fc3c7efaec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:05:51.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.555+0000 7fc3c7efaec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:05:51.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.555+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.566+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:51.571 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.566+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.581 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.576+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:05:51.581 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.576+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.588+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:51.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.588+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.600 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.596+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:05:51.600 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.596+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.604+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.604+0000 7fc3c7efaec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:05:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.604+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.616 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.612+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.612+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.623+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:51.628 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.623+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.634+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:05:51.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.634+0000 7fc3c7efaec0 20 zonegroup default 2019-12-08T15:05:51.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.634+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.639+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.639+0000 7fc3c7efaec0 10 Cannot find current period zone using local zone 2019-12-08T15:05:51.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.639+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.643+0000 7fc3c7efaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:51.648 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.643+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.651+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:51.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.651+0000 7fc3c7efaec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:51.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.659+0000 7fc3c7efaec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:05:51.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.660+0000 7fc3c7efaec0 20 zone default found 2019-12-08T15:05:51.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.660+0000 7fc3c7efaec0 20 started sync module instance, tier type = 2019-12-08T15:05:51.664 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.660+0000 7fc3c7efaec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:05:51.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=0 2019-12-08T15:05:51.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=1 2019-12-08T15:05:51.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=2 2019-12-08T15:05:51.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=3 2019-12-08T15:05:51.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=4 2019-12-08T15:05:51.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=5 2019-12-08T15:05:51.998 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:51.993+0000 7fc3c7efaec0 20 add_watcher() i=6 2019-12-08T15:05:52.043 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:52.038+0000 7fc3c7efaec0 20 add_watcher() i=7 2019-12-08T15:05:52.043 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:52.038+0000 7fc3c7efaec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:05:52.043 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:52.038+0000 7fc385fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:05:52.044 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:52.040+0000 7fc3b5925700 0 -- >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x559427e92810 msgr2=0x559427ec1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T15:05:52.086 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:52.082+0000 7fc3b5124700 0 -- 172.21.15.129:0/1135439596 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7fc2f8022470 msgr2=0x7fc2f8024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:05:53.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.381+0000 7fc2f7fff700 20 reqs_thread_entry: start 2019-12-08T15:05:53.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.381+0000 7fc2f5ffb700 20 reqs_thread_entry: start 2019-12-08T15:05:53.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.381+0000 7fc3c7efaec0 20 init_complete bucket index max shards: 0 2019-12-08T15:05:53.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.382+0000 7fc2e3fff700 20 reqs_thread_entry: start 2019-12-08T15:05:53.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.382+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-08T15:05:53.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.450+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.481+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:53.497 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.493+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.540 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.536+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-08T15:05:53.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.587+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.610 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.606+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:53.621 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.617+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.678 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.674+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-08T15:05:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.746+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.769 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.765+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:53.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.774+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.821+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-08T15:05:53.894 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.890+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.919 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.915+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:53.929 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.924+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:53.972 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:53.968+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-08T15:05:54.018 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.013+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.036 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.031+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.045 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.041+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.088 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.083+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-08T15:05:54.127 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.123+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.145 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.140+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.148+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.214 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.209+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-08T15:05:54.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.296+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.331 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.327+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.348 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.344+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.399 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.395+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-08T15:05:54.478 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.474+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.504 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.499+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.517 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.512+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.567 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.563+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-08T15:05:54.624 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.619+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.647 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.643+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.658 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.654+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.694 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.689+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-08T15:05:54.733 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.729+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.748 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.744+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.756 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.752+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.789 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.784+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-08T15:05:54.833 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.829+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.843+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.857 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.853+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.882+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-08T15:05:54.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.939+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:54.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.954+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:54.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:54.962+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.013 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.008+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-08T15:05:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.068+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.090+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.119 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.115+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.162 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.158+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-08T15:05:55.232 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.228+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.248 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.244+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.256 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.252+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.307 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.303+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-08T15:05:55.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.349+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.366+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.377 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.373+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.417+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-08T15:05:55.462 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.458+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.481 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.477+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.492 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.487+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.522 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.518+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-08T15:05:55.582 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.578+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.600 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.596+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.608+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.651+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-08T15:05:55.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.715+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.742 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.737+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.750+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.791 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.787+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-08T15:05:55.843 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.839+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.858 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.853+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:55.873 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.868+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:55.920 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:55.915+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-08T15:05:56.009 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.005+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.034 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.030+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.051 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.047+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.106 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.101+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-08T15:05:56.166 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.162+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.182 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.178+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.194 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.187+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.225+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-08T15:05:56.268 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.263+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.283 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.278+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.291 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.286+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.330 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.325+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-08T15:05:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.409+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.444 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.440+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.466 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.461+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.520 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.515+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-08T15:05:56.611 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.607+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.632 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.628+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.639+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.684 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.679+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-08T15:05:56.740 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.735+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.760 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.755+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.769 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.765+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.804+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-08T15:05:56.868 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.864+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.887+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:56.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.895+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:56.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.945+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-08T15:05:57.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:56.998+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.023 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.019+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:57.042 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.038+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.082 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.078+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-08T15:05:57.132 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.127+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.156 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.152+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:57.165 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.161+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.201+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-08T15:05:57.279 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.274+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.322 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.312+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:57.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.332+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.402 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.397+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-08T15:05:57.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.457+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.484 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.480+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.489+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.541+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-08T15:05:57.638 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.633+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.667 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.662+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:57.683 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.679+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.725+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-08T15:05:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.772+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.794 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.790+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-08T15:05:57.802 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.798+0000 7fc3c7efaec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-08T15:05:57.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.891+0000 7fc3c7efaec0 20 remove_watcher() i=0 2019-12-08T15:05:57.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.891+0000 7fc3c7efaec0 2 removed watcher, disabling cache 2019-12-08T15:05:57.937 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:57.933+0000 7fc3c7efaec0 20 remove_watcher() i=1 2019-12-08T15:05:58.013 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:58.009+0000 7fc3c7efaec0 20 remove_watcher() i=2 2019-12-08T15:05:58.101 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:58.096+0000 7fc3c7efaec0 20 remove_watcher() i=3 2019-12-08T15:05:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:58.150+0000 7fc3c7efaec0 20 remove_watcher() i=4 2019-12-08T15:05:58.222 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:58.218+0000 7fc3c7efaec0 20 remove_watcher() i=5 2019-12-08T15:05:58.292 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:58.287+0000 7fc3c7efaec0 20 remove_watcher() i=6 2019-12-08T15:05:58.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:58.344+0000 7fc3c7efaec0 20 remove_watcher() i=7 2019-12-08T15:05:58.356 INFO:teuthology.run_tasks:Running task s3tests... 2019-12-08T15:05:58.377 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-12-08T15:05:58.378 INFO:tasks.s3tests:Downloading s3-tests... 2019-12-08T15:05:58.378 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-12-08T15:05:58.378 INFO:teuthology.orchestra.run.smithi129:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-12-08T15:05:58.443 INFO:teuthology.orchestra.run.smithi129.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-12-08T15:05:58.945 INFO:tasks.s3tests:Creating rgw users... 2019-12-08T15:05:58.945 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-12-08T15:05:58.946 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key OWGENUEYNKRCMKZNLSID --secret SZwnTR97+Dzt6MLjNB+ZQ+90fXfrF90b61rFPN+uIsps/hVVapho5A== --email bar.client.0+test@test.test --cluster ceph 2019-12-08T15:05:59.073 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.069+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.094 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.090+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.095 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.090+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.104 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.100+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.104 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.100+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.104+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.108 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.104+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.112+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:59.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.305+0000 7f36bd385ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:05:59.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.305+0000 7f36bd385ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:05:59.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.305+0000 7f36bd385ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:05:59.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.305+0000 7f36bd385ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:05:59.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.305+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.320+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:59.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.320+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.334 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.330+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:05:59.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.330+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.344 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.340+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:59.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.340+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.360+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:05:59.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.360+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.381+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.381+0000 7f36bd385ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:05:59.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.381+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.386+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.386+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.393+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:59.398 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.393+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.423 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.418+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:05:59.423 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.418+0000 7f36bd385ec0 20 zonegroup default 2019-12-08T15:05:59.423 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.418+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.424+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.424+0000 7f36bd385ec0 10 Cannot find current period zone using local zone 2019-12-08T15:05:59.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.424+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.429+0000 7f36bd385ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:05:59.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.429+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.437+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:05:59.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.437+0000 7f36bd385ec0 20 rados->read ofs=0 len=0 2019-12-08T15:05:59.450 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.445+0000 7f36bd385ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:05:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.445+0000 7f36bd385ec0 20 zone default found 2019-12-08T15:05:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.445+0000 7f36bd385ec0 20 started sync module instance, tier type = 2019-12-08T15:05:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.445+0000 7f36bd385ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:05:59.876 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36aa5af700 0 -- 172.21.15.129:0/2294789574 >> [v2:172.21.15.181:6806/10375,v1:172.21.15.181:6809/10375] conn(0x55618b938400 msgr2=0x55618b93a820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:05:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36bd385ec0 20 add_watcher() i=0 2019-12-08T15:05:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36bd385ec0 20 add_watcher() i=1 2019-12-08T15:05:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36bd385ec0 20 add_watcher() i=2 2019-12-08T15:05:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36bd385ec0 20 add_watcher() i=3 2019-12-08T15:05:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36bd385ec0 20 add_watcher() i=4 2019-12-08T15:05:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.872+0000 7f36bd385ec0 20 add_watcher() i=5 2019-12-08T15:05:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.969+0000 7f36bd385ec0 20 add_watcher() i=6 2019-12-08T15:05:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.969+0000 7f36bd385ec0 20 add_watcher() i=7 2019-12-08T15:05:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.969+0000 7f36bd385ec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:05:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.969+0000 7f367afdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:05:59.994 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:05:59.989+0000 7f36aadb0700 0 -- 172.21.15.129:0/4028721893 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x7f35ec002880 msgr2=0x7f35ec004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T15:06:01.348 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.343+0000 7f35e97fa700 20 reqs_thread_entry: start 2019-12-08T15:06:01.348 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.344+0000 7f35e2ffd700 20 reqs_thread_entry: start 2019-12-08T15:06:01.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.344+0000 7f36bd385ec0 20 init_complete bucket index max shards: 0 2019-12-08T15:06:01.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.344+0000 7f35e17fa700 20 reqs_thread_entry: start 2019-12-08T15:06:01.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.344+0000 7f36bd385ec0 20 get_system_obj_state: rctx=0x7ffc58d5fd48 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55618b9da870 s->prefetch_data=0 2019-12-08T15:06:01.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.344+0000 7f36bd385ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-08T15:06:01.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.358+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-08T15:06:01.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.358+0000 7f36bd385ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:01.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.358+0000 7f36bd385ec0 20 get_system_obj_state: rctx=0x7ffc58d5fd58 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x55618b9dc600 s->prefetch_data=0 2019-12-08T15:06:01.363 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.358+0000 7f36bd385ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-08T15:06:01.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.364+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-08T15:06:01.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.364+0000 7f36bd385ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-08T15:06:01.369 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.364+0000 7f36bd385ec0 20 get_system_obj_state: rctx=0x7ffc58d5fd58 obj=default.rgw.meta:users.keys:OWGENUEYNKRCMKZNLSID state=0x55618b9dc600 s->prefetch_data=0 2019-12-08T15:06:01.369 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.364+0000 7f36bd385ec0 10 cache get: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID : miss 2019-12-08T15:06:01.373 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.369+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID info.flags=0x0 2019-12-08T15:06:01.373 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.369+0000 7f36bd385ec0 10 adding default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID to cache LRU end 2019-12-08T15:06:01.373 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.369+0000 7f36bd385ec0 20 get_system_obj_state: rctx=0x7ffc58d5f968 obj=default.rgw.meta:users.keys:OWGENUEYNKRCMKZNLSID state=0x55618b9dc600 s->prefetch_data=0 2019-12-08T15:06:01.374 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.369+0000 7f36bd385ec0 10 cache get: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID : type miss (requested=0x6, cached=0x0) 2019-12-08T15:06:01.378 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.374+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID info.flags=0x0 2019-12-08T15:06:01.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.374+0000 7f36bd385ec0 10 moving default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID to cache LRU end 2019-12-08T15:06:01.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.374+0000 7f36bd385ec0 20 get_system_obj_state: rctx=0x7ffc58d5fae8 obj=default.rgw.meta:users.keys:OWGENUEYNKRCMKZNLSID state=0x55618b9dc600 s->prefetch_data=0 2019-12-08T15:06:01.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.374+0000 7f36bd385ec0 10 cache get: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID : type miss (requested=0x6, cached=0x0) 2019-12-08T15:06:01.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.379+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID info.flags=0x0 2019-12-08T15:06:01.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.379+0000 7f36bd385ec0 10 moving default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID to cache LRU end 2019-12-08T15:06:01.452 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.448+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T15:06:01.452 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.448+0000 7f36bd385ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:01.453 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.448+0000 7f36bd385ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-12-08T15:06:01.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.456+0000 7f368b7fe700 10 RGWWatcher::handle_notify() notify_id 562640715813 cookie 93877441859408 notifier 4664 bl.length()=617 2019-12-08T15:06:01.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.456+0000 7f368b7fe700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T15:06:01.461 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.456+0000 7f368b7fe700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.824+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-08T15:06:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.824+0000 7f36bd385ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-08T15:06:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.824+0000 7f36bd385ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-08T15:06:01.844 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.840+0000 7f368b7fe700 10 RGWWatcher::handle_notify() notify_id 562640715814 cookie 93877441859408 notifier 4664 bl.length()=194 2019-12-08T15:06:01.845 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.840+0000 7f368b7fe700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-08T15:06:01.845 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:01.840+0000 7f368b7fe700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-08T15:06:02.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.066+0000 7f36bd385ec0 10 cache put: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID info.flags=0x7 2019-12-08T15:06:02.071 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.066+0000 7f36bd385ec0 10 moving default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID to cache LRU end 2019-12-08T15:06:02.071 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.066+0000 7f36bd385ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=186 2019-12-08T15:06:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.084+0000 7f368b7fe700 10 RGWWatcher::handle_notify() notify_id 562640715817 cookie 93877441862960 notifier 4664 bl.length()=186 2019-12-08T15:06:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.084+0000 7f368b7fe700 10 cache put: name=default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID info.flags=0x7 2019-12-08T15:06:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.084+0000 7f368b7fe700 10 moving default.rgw.meta+users.keys+OWGENUEYNKRCMKZNLSID to cache LRU end 2019-12-08T15:06:02.370 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T15:06:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "bar.client.0", 2019-12-08T15:06:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. bar.client.0", 2019-12-08T15:06:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "bar.client.0+test@test.test", 2019-12-08T15:06:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T15:06:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T15:06:02.371 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [], 2019-12-08T15:06:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [ 2019-12-08T15:06:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T15:06:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "bar.client.0", 2019-12-08T15:06:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "access_key": "OWGENUEYNKRCMKZNLSID", 2019-12-08T15:06:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "SZwnTR97+Dzt6MLjNB+ZQ+90fXfrF90b61rFPN+uIsps/hVVapho5A==" 2019-12-08T15:06:02.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [], 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T15:06:02.373 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T15:06:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T15:06:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T15:06:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T15:06:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T15:06:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T15:06:02.374 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T15:06:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T15:06:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T15:06:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T15:06:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T15:06:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T15:06:02.375 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T15:06:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T15:06:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T15:06:02.376 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T15:06:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T15:06:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T15:06:02.377 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T15:06:02.377 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:02.426 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.421+0000 7f36bd385ec0 20 remove_watcher() i=0 2019-12-08T15:06:02.426 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.421+0000 7f36bd385ec0 2 removed watcher, disabling cache 2019-12-08T15:06:02.464 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.459+0000 7f36bd385ec0 20 remove_watcher() i=1 2019-12-08T15:06:02.509 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.505+0000 7f36bd385ec0 20 remove_watcher() i=2 2019-12-08T15:06:02.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.539+0000 7f36bd385ec0 20 remove_watcher() i=3 2019-12-08T15:06:02.590 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.585+0000 7f36bd385ec0 20 remove_watcher() i=4 2019-12-08T15:06:02.626 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.622+0000 7f36bd385ec0 20 remove_watcher() i=5 2019-12-08T15:06:02.676 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.672+0000 7f36bd385ec0 20 remove_watcher() i=6 2019-12-08T15:06:02.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.724+0000 7f36bd385ec0 20 remove_watcher() i=7 2019-12-08T15:06:02.736 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 1891351352 --totp-seed PGFCZVKOAQ5ZF5KRADC2A5BPZHRZOTHWHPZ5WRAKO4RRR3ACJXS2RN33IJLQCTEI --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-08T15:06:02.867 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.863+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:02.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.886+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:02.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.886+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:02.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.897+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:02.901 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.897+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:02.907 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.902+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:02.907 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.902+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:02.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:02.911+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:03.216 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.211+0000 7f90f7082ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:06:03.216 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.211+0000 7f90f7082ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:06:03.216 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.211+0000 7f90f7082ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:06:03.216 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.211+0000 7f90f7082ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:06:03.217 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.211+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.234 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.229+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:03.234 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.229+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.242+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:03.247 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.242+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.250+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:03.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.250+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.263+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:03.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.263+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.268+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:03.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.268+0000 7f90f7082ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:06:03.272 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.268+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.273+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:03.277 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.273+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.283+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:03.288 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.283+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.307 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.302+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:03.307 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.302+0000 7f90f7082ec0 20 zonegroup default 2019-12-08T15:06:03.307 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.302+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.306+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.306+0000 7f90f7082ec0 10 Cannot find current period zone using local zone 2019-12-08T15:06:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.306+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.312+0000 7f90f7082ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.312+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.320+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:03.325 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.320+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.328+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.328+0000 7f90f7082ec0 20 zone default found 2019-12-08T15:06:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.328+0000 7f90f7082ec0 20 started sync module instance, tier type = 2019-12-08T15:06:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.328+0000 7f90f7082ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:06:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.701+0000 7f90e42ac700 0 -- 172.21.15.129:0/2106744860 >> [v2:172.21.15.181:6806/10375,v1:172.21.15.181:6809/10375] conn(0x56497d715cd0 msgr2=0x56497d7180f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:06:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=0 2019-12-08T15:06:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=1 2019-12-08T15:06:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=2 2019-12-08T15:06:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=3 2019-12-08T15:06:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=4 2019-12-08T15:06:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=5 2019-12-08T15:06:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.749+0000 7f90f7082ec0 20 add_watcher() i=6 2019-12-08T15:06:03.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.788+0000 7f90f7082ec0 20 add_watcher() i=7 2019-12-08T15:06:03.793 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.788+0000 7f90f7082ec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:06:03.793 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.788+0000 7f90b4fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:06:03.808 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:03.804+0000 7f90e3aab700 0 -- 172.21.15.129:0/1951135445 >> v2:172.21.15.181:3300/0 conn(0x56497d72f660 msgr2=0x56497d731b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:06:05.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.252+0000 7f9026ffd700 20 reqs_thread_entry: start 2019-12-08T15:06:05.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.253+0000 7f9024ff9700 20 reqs_thread_entry: start 2019-12-08T15:06:05.257 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.253+0000 7f90f7082ec0 20 init_complete bucket index max shards: 0 2019-12-08T15:06:05.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.253+0000 7f901affd700 20 reqs_thread_entry: start 2019-12-08T15:06:05.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.253+0000 7f90f7082ec0 20 get_system_obj_state: rctx=0x7ffc248a50c8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x56497d7a71d0 s->prefetch_data=0 2019-12-08T15:06:05.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.253+0000 7f90f7082ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-08T15:06:05.270 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.265+0000 7f90f7082ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-08T15:06:05.270 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.265+0000 7f90f7082ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:05.270 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.265+0000 7f90f7082ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T15:06:05.270 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.265+0000 7f90f7082ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T15:06:05.271 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.265+0000 7f90f7082ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T15:06:05.271 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.265+0000 7f90f7082ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:05.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.294+0000 7f90f7082ec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-08T15:06:05.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.294+0000 7f90f7082ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-08T15:06:05.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:05.294+0000 7f90f7082ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:07.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:07.723+0000 7f90f7082ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T15:06:07.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:07.723+0000 7f90f7082ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:07.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:07.723+0000 7f90f7082ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-12-08T15:06:07.735 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:07.730+0000 7f90d9ffb700 10 RGWWatcher::handle_notify() notify_id 571230650407 cookie 94873637121760 notifier 4673 bl.length()=631 2019-12-08T15:06:07.735 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:07.730+0000 7f90d9ffb700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T15:06:07.735 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:07.730+0000 7f90d9ffb700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T15:06:08.121 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.115+0000 7f90f7082ec0 20 remove_watcher() i=0 2019-12-08T15:06:08.121 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.115+0000 7f90f7082ec0 2 removed watcher, disabling cache 2019-12-08T15:06:08.192 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.187+0000 7f90f7082ec0 20 remove_watcher() i=1 2019-12-08T15:06:08.471 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.465+0000 7f90f7082ec0 20 remove_watcher() i=2 2019-12-08T15:06:08.511 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.505+0000 7f90f7082ec0 20 remove_watcher() i=3 2019-12-08T15:06:08.690 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.685+0000 7f90f7082ec0 20 remove_watcher() i=4 2019-12-08T15:06:08.752 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.747+0000 7f90f7082ec0 20 remove_watcher() i=5 2019-12-08T15:06:08.799 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.794+0000 7f90f7082ec0 20 remove_watcher() i=6 2019-12-08T15:06:08.843 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:08.838+0000 7f90f7082ec0 20 remove_watcher() i=7 2019-12-08T15:06:08.850 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-12-08T15:06:08.851 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key YZGSEIIYRKKCPAVLVUEL --secret d27v+0EHuKyHTju2jP506p4Np5peapEb/dLKgplN5rmfkVlKZEvO4Q== --email foo.client.0+test@test.test --cluster ceph 2019-12-08T15:06:09.007 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.002+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.035+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.035+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.048 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.042+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.048 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.042+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.052 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.047+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.052 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.047+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.059 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.054+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:09.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.240+0000 7faf33731ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:06:09.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.240+0000 7faf33731ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:06:09.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.240+0000 7faf33731ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:06:09.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.240+0000 7faf33731ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:06:09.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.240+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.250+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.250+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.258+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:09.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.259+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.275 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.270+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:09.275 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.270+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.294+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:09.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.294+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.303+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.303+0000 7faf33731ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:06:09.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.303+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.316+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.316+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.328+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:09.333 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.328+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.352 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.339+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:09.352 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.339+0000 7faf33731ec0 20 zonegroup default 2019-12-08T15:06:09.352 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.339+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.345+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.345+0000 7faf33731ec0 10 Cannot find current period zone using local zone 2019-12-08T15:06:09.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.345+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.351+0000 7faf33731ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:09.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.351+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.359+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:09.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.359+0000 7faf33731ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:09.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.365+0000 7faf33731ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:09.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.365+0000 7faf33731ec0 20 zone default found 2019-12-08T15:06:09.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.365+0000 7faf33731ec0 20 started sync module instance, tier type = 2019-12-08T15:06:09.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.365+0000 7faf33731ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:06:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.691+0000 7faf33731ec0 20 add_watcher() i=0 2019-12-08T15:06:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.691+0000 7faf33731ec0 20 add_watcher() i=1 2019-12-08T15:06:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.691+0000 7faf33731ec0 20 add_watcher() i=2 2019-12-08T15:06:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.691+0000 7faf33731ec0 20 add_watcher() i=3 2019-12-08T15:06:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.691+0000 7faf33731ec0 20 add_watcher() i=4 2019-12-08T15:06:09.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.691+0000 7faf33731ec0 20 add_watcher() i=5 2019-12-08T15:06:09.709 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.703+0000 7faf33731ec0 20 add_watcher() i=6 2019-12-08T15:06:09.741 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.736+0000 7faf33731ec0 20 add_watcher() i=7 2019-12-08T15:06:09.742 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.736+0000 7faf33731ec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:06:09.742 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.736+0000 7faef0fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:06:09.743 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.738+0000 7faf2115c700 0 -- >> v2:172.21.15.129:3300/0 conn(0x561af6841c20 msgr2=0x561af6842030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-08T15:06:09.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:09.761+0000 7faf2095b700 0 -- 172.21.15.129:0/3371931631 >> v2:172.21.15.181:3300/0 conn(0x561af684c6c0 msgr2=0x561af684eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:06:11.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.303+0000 7fae62ffd700 20 reqs_thread_entry: start 2019-12-08T15:06:11.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.303+0000 7fae60ff9700 20 reqs_thread_entry: start 2019-12-08T15:06:11.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.303+0000 7faf33731ec0 20 init_complete bucket index max shards: 0 2019-12-08T15:06:11.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.303+0000 7fae56ffd700 20 reqs_thread_entry: start 2019-12-08T15:06:11.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.304+0000 7faf33731ec0 20 get_system_obj_state: rctx=0x7ffc7090c4e8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x561af68c40d0 s->prefetch_data=0 2019-12-08T15:06:11.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.304+0000 7faf33731ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-08T15:06:11.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.330+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-08T15:06:11.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.330+0000 7faf33731ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:11.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.330+0000 7faf33731ec0 20 get_system_obj_state: rctx=0x7ffc7090c4f8 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x561af68c5e60 s->prefetch_data=0 2019-12-08T15:06:11.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.330+0000 7faf33731ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-08T15:06:11.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.335+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-08T15:06:11.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.335+0000 7faf33731ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-08T15:06:11.340 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.335+0000 7faf33731ec0 20 get_system_obj_state: rctx=0x7ffc7090c4f8 obj=default.rgw.meta:users.keys:YZGSEIIYRKKCPAVLVUEL state=0x561af68c5e60 s->prefetch_data=0 2019-12-08T15:06:11.341 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.335+0000 7faf33731ec0 10 cache get: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL : miss 2019-12-08T15:06:11.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.339+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL info.flags=0x0 2019-12-08T15:06:11.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.339+0000 7faf33731ec0 10 adding default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL to cache LRU end 2019-12-08T15:06:11.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.340+0000 7faf33731ec0 20 get_system_obj_state: rctx=0x7ffc7090c108 obj=default.rgw.meta:users.keys:YZGSEIIYRKKCPAVLVUEL state=0x561af68c5e60 s->prefetch_data=0 2019-12-08T15:06:11.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.340+0000 7faf33731ec0 10 cache get: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL : type miss (requested=0x6, cached=0x0) 2019-12-08T15:06:11.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.345+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL info.flags=0x0 2019-12-08T15:06:11.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.345+0000 7faf33731ec0 10 moving default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL to cache LRU end 2019-12-08T15:06:11.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.345+0000 7faf33731ec0 20 get_system_obj_state: rctx=0x7ffc7090c288 obj=default.rgw.meta:users.keys:YZGSEIIYRKKCPAVLVUEL state=0x561af68c5e60 s->prefetch_data=0 2019-12-08T15:06:11.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.345+0000 7faf33731ec0 10 cache get: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL : type miss (requested=0x6, cached=0x0) 2019-12-08T15:06:11.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.350+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL info.flags=0x0 2019-12-08T15:06:11.355 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.350+0000 7faf33731ec0 10 moving default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL to cache LRU end 2019-12-08T15:06:11.429 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.424+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T15:06:11.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.424+0000 7faf33731ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:11.430 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.424+0000 7faf33731ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-12-08T15:06:11.456 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.451+0000 7faf09ffb700 10 RGWWatcher::handle_notify() notify_id 575525617692 cookie 94673804898496 notifier 4980 bl.length()=617 2019-12-08T15:06:11.457 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.451+0000 7faf09ffb700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T15:06:11.457 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:11.451+0000 7faf09ffb700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:12.349 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.344+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-08T15:06:12.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.344+0000 7faf33731ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-08T15:06:12.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.344+0000 7faf33731ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-08T15:06:12.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.362+0000 7faf09ffb700 10 RGWWatcher::handle_notify() notify_id 575525617731 cookie 94673804926560 notifier 4980 bl.length()=194 2019-12-08T15:06:12.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.362+0000 7faf09ffb700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-08T15:06:12.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.362+0000 7faf09ffb700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-08T15:06:12.722 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.717+0000 7faf33731ec0 10 cache put: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL info.flags=0x7 2019-12-08T15:06:12.722 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.717+0000 7faf33731ec0 10 moving default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL to cache LRU end 2019-12-08T15:06:12.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.717+0000 7faf33731ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=186 2019-12-08T15:06:12.736 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.731+0000 7faf09ffb700 10 RGWWatcher::handle_notify() notify_id 575525617732 cookie 94673804926560 notifier 4980 bl.length()=186 2019-12-08T15:06:12.736 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.731+0000 7faf09ffb700 10 cache put: name=default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL info.flags=0x7 2019-12-08T15:06:12.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:12.731+0000 7faf09ffb700 10 moving default.rgw.meta+users.keys+YZGSEIIYRKKCPAVLVUEL to cache LRU end 2019-12-08T15:06:12.987 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T15:06:12.987 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "foo.client.0", 2019-12-08T15:06:12.989 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. foo.client.0", 2019-12-08T15:06:12.989 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "foo.client.0+test@test.test", 2019-12-08T15:06:12.989 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T15:06:12.990 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T15:06:12.990 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [], 2019-12-08T15:06:12.990 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [ 2019-12-08T15:06:12.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T15:06:12.990 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "foo.client.0", 2019-12-08T15:06:12.990 INFO:teuthology.orchestra.run.smithi129.stdout: "access_key": "YZGSEIIYRKKCPAVLVUEL", 2019-12-08T15:06:12.991 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "d27v+0EHuKyHTju2jP506p4Np5peapEb/dLKgplN5rmfkVlKZEvO4Q==" 2019-12-08T15:06:12.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T15:06:12.991 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T15:06:12.991 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [], 2019-12-08T15:06:12.991 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T15:06:12.992 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T15:06:12.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T15:06:12.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T15:06:12.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T15:06:12.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T15:06:12.993 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T15:06:12.994 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T15:06:12.995 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T15:06:12.995 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T15:06:12.995 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T15:06:12.995 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:13.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.053+0000 7faf33731ec0 20 remove_watcher() i=0 2019-12-08T15:06:13.059 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.053+0000 7faf33731ec0 2 removed watcher, disabling cache 2019-12-08T15:06:13.116 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.111+0000 7faf33731ec0 20 remove_watcher() i=1 2019-12-08T15:06:13.157 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.152+0000 7faf33731ec0 20 remove_watcher() i=2 2019-12-08T15:06:13.218 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.213+0000 7faf33731ec0 20 remove_watcher() i=3 2019-12-08T15:06:13.262 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.257+0000 7faf33731ec0 20 remove_watcher() i=4 2019-12-08T15:06:13.311 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.306+0000 7faf33731ec0 20 remove_watcher() i=5 2019-12-08T15:06:13.357 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.352+0000 7faf33731ec0 20 remove_watcher() i=6 2019-12-08T15:06:13.410 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.405+0000 7faf33731ec0 20 remove_watcher() i=7 2019-12-08T15:06:13.417 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 9944127061 --totp-seed LCIQQJW5BTPAYVHCCL73ATZIZBN5VFS4IGINNAUWYTPZB3FBMMXTJKJBYQ5H5A7U --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-08T15:06:13.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.557+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.615 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.608+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.615 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.609+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.622 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.617+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.622 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.617+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.626 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.621+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.627 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.621+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.630+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.795+0000 7fb344925ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:06:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.795+0000 7fb344925ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:06:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.795+0000 7fb344925ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:06:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.795+0000 7fb344925ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:06:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.795+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.804+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:13.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.804+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.813+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:13.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.813+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.821+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:13.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.821+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.837+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:13.843 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.837+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.842+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.842+0000 7fb344925ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:06:13.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.842+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.852 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.846+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.852 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.846+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.854+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:13.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.854+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.868+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:13.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.868+0000 7fb344925ec0 20 zonegroup default 2019-12-08T15:06:13.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.868+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.878 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.873+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.878 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.873+0000 7fb344925ec0 10 Cannot find current period zone using local zone 2019-12-08T15:06:13.879 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.873+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.878+0000 7fb344925ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:13.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.878+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.891 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.885+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:13.891 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.885+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:13.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.893+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:13.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.893+0000 7fb344925ec0 20 zone default found 2019-12-08T15:06:13.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.893+0000 7fb344925ec0 20 started sync module instance, tier type = 2019-12-08T15:06:13.899 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:13.893+0000 7fb344925ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:06:14.286 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.281+0000 7fb344925ec0 20 add_watcher() i=0 2019-12-08T15:06:14.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.281+0000 7fb344925ec0 20 add_watcher() i=1 2019-12-08T15:06:14.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.281+0000 7fb344925ec0 20 add_watcher() i=2 2019-12-08T15:06:14.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.281+0000 7fb344925ec0 20 add_watcher() i=3 2019-12-08T15:06:14.287 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.281+0000 7fb344925ec0 20 add_watcher() i=4 2019-12-08T15:06:14.288 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.281+0000 7fb344925ec0 20 add_watcher() i=5 2019-12-08T15:06:14.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.309+0000 7fb344925ec0 20 add_watcher() i=6 2019-12-08T15:06:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.344+0000 7fb344925ec0 20 add_watcher() i=7 2019-12-08T15:06:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.344+0000 7fb344925ec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:06:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.345+0000 7fb3027dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:06:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.345+0000 7fb33134e700 0 -- >> v2:172.21.15.181:3300/0 conn(0x55fe4ae7fb50 msgr2=0x55fe4ae82060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-08T15:06:14.408 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:14.403+0000 7fb331b4f700 0 -- 172.21.15.129:0/1986874716 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7fb274022470 msgr2=0x7fb274024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-08T15:06:15.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.855+0000 7fb278ff9700 20 reqs_thread_entry: start 2019-12-08T15:06:15.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.855+0000 7fb2727fc700 20 reqs_thread_entry: start 2019-12-08T15:06:15.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.855+0000 7fb344925ec0 20 init_complete bucket index max shards: 0 2019-12-08T15:06:15.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.856+0000 7fb270ff9700 20 reqs_thread_entry: start 2019-12-08T15:06:15.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.856+0000 7fb344925ec0 20 get_system_obj_state: rctx=0x7ffed63bc838 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55fe4aefd8a0 s->prefetch_data=0 2019-12-08T15:06:15.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.856+0000 7fb344925ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-08T15:06:15.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.877+0000 7fb344925ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-08T15:06:15.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.877+0000 7fb344925ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:15.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.877+0000 7fb344925ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T15:06:15.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.877+0000 7fb344925ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T15:06:15.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.877+0000 7fb344925ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T15:06:15.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.877+0000 7fb344925ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:15.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.893+0000 7fb344925ec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-08T15:06:15.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.893+0000 7fb344925ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-08T15:06:15.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:15.893+0000 7fb344925ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:16.053 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.045+0000 7fb344925ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T15:06:16.053 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.045+0000 7fb344925ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:16.053 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.045+0000 7fb344925ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-12-08T15:06:16.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.054+0000 7fb312ffd700 10 RGWWatcher::handle_notify() notify_id 575525617693 cookie 94550666648432 notifier 4903 bl.length()=631 2019-12-08T15:06:16.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.054+0000 7fb312ffd700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T15:06:16.058 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.054+0000 7fb312ffd700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T15:06:16.465 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.460+0000 7fb344925ec0 20 remove_watcher() i=0 2019-12-08T15:06:16.465 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.460+0000 7fb344925ec0 2 removed watcher, disabling cache 2019-12-08T15:06:16.507 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.502+0000 7fb344925ec0 20 remove_watcher() i=1 2019-12-08T15:06:16.548 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.543+0000 7fb344925ec0 20 remove_watcher() i=2 2019-12-08T15:06:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.603+0000 7fb344925ec0 20 remove_watcher() i=3 2019-12-08T15:06:16.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.652+0000 7fb344925ec0 20 remove_watcher() i=4 2019-12-08T15:06:16.711 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.706+0000 7fb344925ec0 20 remove_watcher() i=5 2019-12-08T15:06:16.760 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.755+0000 7fb344925ec0 20 remove_watcher() i=6 2019-12-08T15:06:16.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.810+0000 7fb344925ec0 20 remove_watcher() i=7 2019-12-08T15:06:16.822 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-12-08T15:06:16.823 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key TUAVGJGNDIUFSGEVIIJK --secret LJnZ7VKWOspOyvgDCXscGl/uEywqXruzuVesj9PkLVG2/BJEVQmlPQ== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-12-08T15:06:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.954+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:16.987 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.983+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:16.988 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.983+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:16.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.993+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:16.997 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.993+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.999+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:17.003 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:16.999+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.010 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.006+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:17.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.226+0000 7fe82525fec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:06:17.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.226+0000 7fe82525fec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:06:17.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.226+0000 7fe82525fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:06:17.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.226+0000 7fe82525fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:06:17.231 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.226+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.241+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:17.245 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.241+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.253+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:17.258 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.253+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.269 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.265+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:17.269 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.265+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.288 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.284+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:17.289 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.284+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.295 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.291+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:17.296 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.291+0000 7fe82525fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:06:17.296 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.291+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.303 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.299+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:17.304 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.299+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.314 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.310+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:17.315 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.310+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.327+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:17.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.328+0000 7fe82525fec0 20 zonegroup default 2019-12-08T15:06:17.332 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.328+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.339 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.335+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:17.339 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.335+0000 7fe82525fec0 10 Cannot find current period zone using local zone 2019-12-08T15:06:17.339 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.335+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.341+0000 7fe82525fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:17.345 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.341+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.349+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:17.354 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.349+0000 7fe82525fec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:17.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.360+0000 7fe82525fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:17.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.360+0000 7fe82525fec0 20 zone default found 2019-12-08T15:06:17.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.360+0000 7fe82525fec0 20 started sync module instance, tier type = 2019-12-08T15:06:17.364 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.360+0000 7fe82525fec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:06:17.777 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=0 2019-12-08T15:06:17.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=1 2019-12-08T15:06:17.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=2 2019-12-08T15:06:17.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=3 2019-12-08T15:06:17.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=4 2019-12-08T15:06:17.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=5 2019-12-08T15:06:17.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.773+0000 7fe82525fec0 20 add_watcher() i=6 2019-12-08T15:06:17.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.806+0000 7fe82525fec0 20 add_watcher() i=7 2019-12-08T15:06:17.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.806+0000 7fe82525fec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:06:17.811 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.807+0000 7fe7e2fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:06:17.817 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:17.812+0000 7fe812489700 0 -- 172.21.15.129:0/1957953116 >> v2:172.21.15.181:3300/0 conn(0x561b803c9050 msgr2=0x561b803cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T15:06:18.019 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:18.015+0000 7fe812489700 0 --2- 172.21.15.129:0/1957953116 >> v2:172.21.15.181:3300/0 conn(0x561b803c9050 0x561b803cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-08T15:06:18.070 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:18.066+0000 7fe812c8a700 0 -- 172.21.15.129:0/3321595913 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7fe754022480 msgr2=0x7fe754024970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:06:19.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.765+0000 7fe7517fa700 20 reqs_thread_entry: start 2019-12-08T15:06:19.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.765+0000 7fe74affd700 20 reqs_thread_entry: start 2019-12-08T15:06:19.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.765+0000 7fe82525fec0 20 init_complete bucket index max shards: 0 2019-12-08T15:06:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.766+0000 7fe7497fa700 20 reqs_thread_entry: start 2019-12-08T15:06:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.766+0000 7fe82525fec0 20 get_system_obj_state: rctx=0x7ffc42151948 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x561b8044aca0 s->prefetch_data=0 2019-12-08T15:06:19.771 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.766+0000 7fe82525fec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-08T15:06:19.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.779+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-12-08T15:06:19.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.779+0000 7fe82525fec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:19.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.779+0000 7fe82525fec0 20 get_system_obj_state: rctx=0x7ffc42151958 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x561b8044ca50 s->prefetch_data=0 2019-12-08T15:06:19.783 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.779+0000 7fe82525fec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-12-08T15:06:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.788+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-12-08T15:06:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.788+0000 7fe82525fec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-08T15:06:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.788+0000 7fe82525fec0 20 get_system_obj_state: rctx=0x7ffc42151958 obj=default.rgw.meta:users.keys:TUAVGJGNDIUFSGEVIIJK state=0x561b8044ca50 s->prefetch_data=0 2019-12-08T15:06:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.788+0000 7fe82525fec0 10 cache get: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK : miss 2019-12-08T15:06:19.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.802+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK info.flags=0x0 2019-12-08T15:06:19.806 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.802+0000 7fe82525fec0 10 adding default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK to cache LRU end 2019-12-08T15:06:19.807 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.802+0000 7fe82525fec0 20 get_system_obj_state: rctx=0x7ffc42151568 obj=default.rgw.meta:users.keys:TUAVGJGNDIUFSGEVIIJK state=0x561b8044ca50 s->prefetch_data=0 2019-12-08T15:06:19.807 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.802+0000 7fe82525fec0 10 cache get: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK : type miss (requested=0x6, cached=0x0) 2019-12-08T15:06:19.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.810+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK info.flags=0x0 2019-12-08T15:06:19.814 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.810+0000 7fe82525fec0 10 moving default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK to cache LRU end 2019-12-08T15:06:19.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.810+0000 7fe82525fec0 20 get_system_obj_state: rctx=0x7ffc421516e8 obj=default.rgw.meta:users.keys:TUAVGJGNDIUFSGEVIIJK state=0x561b8044ca50 s->prefetch_data=0 2019-12-08T15:06:19.815 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.810+0000 7fe82525fec0 10 cache get: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK : type miss (requested=0x6, cached=0x0) 2019-12-08T15:06:19.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.815+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK info.flags=0x0 2019-12-08T15:06:19.820 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.815+0000 7fe82525fec0 10 moving default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK to cache LRU end 2019-12-08T15:06:19.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.885+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-08T15:06:19.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.885+0000 7fe82525fec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:19.889 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.885+0000 7fe82525fec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-12-08T15:06:19.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.898+0000 7fe7f37fe700 10 RGWWatcher::handle_notify() notify_id 575525617718 cookie 94676115477664 notifier 5013 bl.length()=691 2019-12-08T15:06:19.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.898+0000 7fe7f37fe700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-08T15:06:19.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:19.898+0000 7fe7f37fe700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:20.020 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.016+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-08T15:06:20.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.016+0000 7fe82525fec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-08T15:06:20.021 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.016+0000 7fe82525fec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-12-08T15:06:20.039 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.035+0000 7fe7f37fe700 10 RGWWatcher::handle_notify() notify_id 575525617733 cookie 94676115495296 notifier 5013 bl.length()=224 2019-12-08T15:06:20.039 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.035+0000 7fe7f37fe700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-08T15:06:20.040 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.035+0000 7fe7f37fe700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-08T15:06:20.488 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.483+0000 7fe82525fec0 10 cache put: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK info.flags=0x7 2019-12-08T15:06:20.488 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.483+0000 7fe82525fec0 10 moving default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK to cache LRU end 2019-12-08T15:06:20.488 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.483+0000 7fe82525fec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=201 2019-12-08T15:06:20.510 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.505+0000 7fe7f37fe700 10 RGWWatcher::handle_notify() notify_id 575525617704 cookie 94676115453712 notifier 5013 bl.length()=201 2019-12-08T15:06:20.510 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.505+0000 7fe7f37fe700 10 cache put: name=default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK info.flags=0x7 2019-12-08T15:06:20.510 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.505+0000 7fe7f37fe700 10 moving default.rgw.meta+users.keys+TUAVGJGNDIUFSGEVIIJK to cache LRU end 2019-12-08T15:06:20.548 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T15:06:20.549 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "testx$tenanteduser.client.0", 2019-12-08T15:06:20.549 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-12-08T15:06:20.549 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-12-08T15:06:20.549 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T15:06:20.549 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T15:06:20.549 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [], 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [ 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "testx$tenanteduser.client.0", 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: "access_key": "TUAVGJGNDIUFSGEVIIJK", 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "LJnZ7VKWOspOyvgDCXscGl/uEywqXruzuVesj9PkLVG2/BJEVQmlPQ==" 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T15:06:20.550 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [], 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T15:06:20.551 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T15:06:20.552 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T15:06:20.553 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T15:06:20.554 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T15:06:20.554 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:20.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.631+0000 7fe82525fec0 20 remove_watcher() i=0 2019-12-08T15:06:20.636 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.631+0000 7fe82525fec0 2 removed watcher, disabling cache 2019-12-08T15:06:20.668 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.664+0000 7fe82525fec0 20 remove_watcher() i=1 2019-12-08T15:06:20.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.703+0000 7fe82525fec0 20 remove_watcher() i=2 2019-12-08T15:06:20.767 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.763+0000 7fe82525fec0 20 remove_watcher() i=3 2019-12-08T15:06:20.803 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.798+0000 7fe82525fec0 20 remove_watcher() i=4 2019-12-08T15:06:20.859 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.854+0000 7fe82525fec0 20 remove_watcher() i=5 2019-12-08T15:06:20.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.893+0000 7fe82525fec0 20 remove_watcher() i=6 2019-12-08T15:06:20.952 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:20.948+0000 7fe82525fec0 20 remove_watcher() i=7 2019-12-08T15:06:20.959 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 7723179381 --totp-seed DYXUX6SIBJKRB4FMWU36E7XQDWH2NYEJIZF5IPVHCZAN6PABNEFJVBRDQNTSV7VZ --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-08T15:06:21.080 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.076+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.104 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.100+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.104 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.100+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.117 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.113+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.118 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.113+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.122 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.118+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.122 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.118+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.129 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.125+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:21.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.348+0000 7f4fc8698ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T15:06:21.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.348+0000 7f4fc8698ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T15:06:21.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.348+0000 7f4fc8698ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T15:06:21.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.348+0000 7f4fc8698ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T15:06:21.353 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.348+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.358+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:21.362 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.358+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.365+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:21.370 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.366+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.378+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:21.383 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.378+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.407 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.403+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:21.408 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.403+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.418 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.413+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.418 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.413+0000 7f4fc8698ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T15:06:21.418 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.413+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.423+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.423+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.441 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.437+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:21.442 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.437+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.449 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.444+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T15:06:21.449 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.444+0000 7f4fc8698ec0 20 zonegroup default 2019-12-08T15:06:21.449 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.444+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.451+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.455 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.451+0000 7f4fc8698ec0 10 Cannot find current period zone using local zone 2019-12-08T15:06:21.456 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.451+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.458+0000 7f4fc8698ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T15:06:21.463 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.458+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.470+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T15:06:21.474 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.470+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:21.486 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.482+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T15:06:21.487 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.482+0000 7f4fc8698ec0 20 zone default found 2019-12-08T15:06:21.487 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.482+0000 7f4fc8698ec0 20 started sync module instance, tier type = 2019-12-08T15:06:21.487 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.482+0000 7f4fc8698ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T15:06:21.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=0 2019-12-08T15:06:21.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=1 2019-12-08T15:06:21.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=2 2019-12-08T15:06:21.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=3 2019-12-08T15:06:21.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=4 2019-12-08T15:06:21.848 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=5 2019-12-08T15:06:21.849 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.843+0000 7f4fc8698ec0 20 add_watcher() i=6 2019-12-08T15:06:21.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.883+0000 7f4fc8698ec0 20 add_watcher() i=7 2019-12-08T15:06:21.887 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.883+0000 7f4fc8698ec0 2 all 8 watchers are set, enabling cache 2019-12-08T15:06:21.888 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.883+0000 7f4f867dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T15:06:21.902 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.898+0000 7f4fb50c1700 0 -- >> v1:172.21.15.129:6789/0 conn(0x55b1fe746d10 legacy=0x55b1fe7491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-08T15:06:21.979 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:21.975+0000 7f4fb60c3700 0 -- 172.21.15.129:0/891931651 >> [v2:172.21.15.129:6802/10195,v1:172.21.15.129:6803/10195] conn(0x55b1fe755dd0 msgr2=0x55b1fe7581f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T15:06:23.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.746+0000 7f4efcff9700 20 reqs_thread_entry: start 2019-12-08T15:06:23.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.746+0000 7f4ef67fc700 20 reqs_thread_entry: start 2019-12-08T15:06:23.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.746+0000 7f4fc8698ec0 20 init_complete bucket index max shards: 0 2019-12-08T15:06:23.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.746+0000 7f4ef4ff9700 20 reqs_thread_entry: start 2019-12-08T15:06:23.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.746+0000 7f4fc8698ec0 20 get_system_obj_state: rctx=0x7fffd9eb3928 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55b1fe7bce70 s->prefetch_data=0 2019-12-08T15:06:23.752 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.746+0000 7f4fc8698ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-08T15:06:23.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.760+0000 7f4fc8698ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-08T15:06:23.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.760+0000 7f4fc8698ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:23.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.760+0000 7f4fc8698ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T15:06:23.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.760+0000 7f4fc8698ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T15:06:23.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.760+0000 7f4fc8698ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T15:06:23.765 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.760+0000 7f4fc8698ec0 20 rados->read ofs=0 len=0 2019-12-08T15:06:23.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.774+0000 7f4fc8698ec0 20 rados_obj.operate() r=0 bl.length=491 2019-12-08T15:06:23.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.774+0000 7f4fc8698ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-08T15:06:23.778 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.774+0000 7f4fc8698ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:23.932 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.928+0000 7f4fc8698ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-08T15:06:23.933 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.928+0000 7f4fc8698ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:23.933 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.928+0000 7f4fc8698ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-12-08T15:06:23.950 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.946+0000 7f4f96ffd700 10 RGWWatcher::handle_notify() notify_id 575525617719 cookie 94222966493360 notifier 4697 bl.length()=705 2019-12-08T15:06:23.950 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.946+0000 7f4f96ffd700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-08T15:06:23.951 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:23.946+0000 7f4f96ffd700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T15:06:24.450 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.446+0000 7f4fc8698ec0 20 remove_watcher() i=0 2019-12-08T15:06:24.450 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.446+0000 7f4fc8698ec0 2 removed watcher, disabling cache 2019-12-08T15:06:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.501+0000 7f4fc8698ec0 20 remove_watcher() i=1 2019-12-08T15:06:24.544 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.540+0000 7f4fc8698ec0 20 remove_watcher() i=2 2019-12-08T15:06:24.623 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.619+0000 7f4fc8698ec0 20 remove_watcher() i=3 2019-12-08T15:06:24.670 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.666+0000 7f4fc8698ec0 20 remove_watcher() i=4 2019-12-08T15:06:24.722 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.718+0000 7f4fc8698ec0 20 remove_watcher() i=5 2019-12-08T15:06:24.771 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.766+0000 7f4fc8698ec0 20 remove_watcher() i=6 2019-12-08T15:06:24.827 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T15:06:24.823+0000 7f4fc8698ec0 20 remove_watcher() i=7 2019-12-08T15:06:24.834 INFO:tasks.s3tests:Configuring s3-tests... 2019-12-08T15:06:24.835 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-12-08T15:06:30.724 INFO:teuthology.orchestra.run.smithi129.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-08T15:06:30.834 INFO:teuthology.orchestra.run.smithi129.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-08T15:06:30.871 INFO:teuthology.orchestra.run.smithi129.stdout:Loading mirror speeds from cached hostfile 2019-12-08T15:06:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: * base: mirrors.umflint.edu 2019-12-08T15:06:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-08T15:06:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: * extras: mirror.facebook.net 2019-12-08T15:06:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: * updates: mirror.math.princeton.edu 2019-12-08T15:06:31.641 INFO:teuthology.orchestra.run.smithi129.stdout:7 packages excluded due to repository priority protections 2019-12-08T15:06:31.999 INFO:teuthology.orchestra.run.smithi129.stdout:Resolving Dependencies 2019-12-08T15:06:31.999 INFO:teuthology.orchestra.run.smithi129.stdout:--> Running transaction check 2019-12-08T15:06:31.999 INFO:teuthology.orchestra.run.smithi129.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-12-08T15:06:32.400 INFO:teuthology.orchestra.run.smithi129.stdout:--> Finished Dependency Resolution 2019-12-08T15:06:32.642 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:32.642 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies Resolved 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-12-08T15:06:32.643 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 23 k 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 27 k 2019-12-08T15:06:32.644 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading packages: 2019-12-08T15:06:32.784 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2019-12-08T15:06:32.784 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2019-12-08T15:06:32.784 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded 2019-12-08T15:06:32.784 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2019-12-08T15:06:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-08T15:06:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-08T15:06:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:33.773 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2019-12-08T15:06:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-12-08T15:06:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:33.773 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2019-12-08T15:06:33.980 INFO:teuthology.orchestra.run.smithi129.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-12-08T15:06:35.271 INFO:teuthology.orchestra.run.smithi129.stdout:Installing setuptools, pip, wheel...done. 2019-12-08T15:06:35.521 INFO:teuthology.orchestra.run.smithi129.stderr:Cache entry deserialization failed, entry ignored 2019-12-08T15:06:35.658 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting pip 2019-12-08T15:06:35.744 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-08T15:06:35.809 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: pip 2019-12-08T15:06:35.809 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: pip 9.0.1 2019-12-08T15:06:35.871 INFO:teuthology.orchestra.run.smithi129.stdout: Uninstalling pip-9.0.1: 2019-12-08T15:06:35.914 INFO:teuthology.orchestra.run.smithi129.stdout: Successfully uninstalled pip-9.0.1 2019-12-08T15:06:36.503 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed pip-19.3.1 2019-12-08T15:06:36.779 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T15:06:37.218 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting setuptools==32.3.1 2019-12-08T15:06:37.221 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-08T15:06:37.261 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: setuptools 2019-12-08T15:06:37.262 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: setuptools 28.8.0 2019-12-08T15:06:37.280 INFO:teuthology.orchestra.run.smithi129.stdout: Uninstalling setuptools-28.8.0: 2019-12-08T15:06:37.289 INFO:teuthology.orchestra.run.smithi129.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-08T15:06:37.414 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed setuptools-32.3.1 2019-12-08T15:06:37.846 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T15:06:38.029 INFO:teuthology.orchestra.run.smithi129.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-08T15:06:38.082 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting nose>=1.0.0 2019-12-08T15:06:38.085 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-08T15:06:38.181 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting boto>=2.6.0 2019-12-08T15:06:38.187 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-08T15:06:38.705 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting boto3>=1.0.0 2019-12-08T15:06:38.746 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/1f/a5/6b25e39aea40bc03163615c07572d90aaddc9951f5af9c21204d3da46398/boto3-1.10.34-py2.py3-none-any.whl (128kB) 2019-12-08T15:06:38.853 INFO:teuthology.orchestra.run.smithi129.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-08T15:06:39.127 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting gevent>=1.0 2019-12-08T15:06:39.140 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T15:06:39.395 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting isodate>=0.4.4 2019-12-08T15:06:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-12-08T15:06:39.563 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting requests>=0.14.0 2019-12-08T15:06:39.572 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-12-08T15:06:39.824 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting pytz>=2011k 2019-12-08T15:06:39.835 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-12-08T15:06:40.072 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting ordereddict 2019-12-08T15:06:40.082 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-12-08T15:06:40.386 INFO:teuthology.orchestra.run.smithi129.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-12-08T15:06:40.788 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting lxml 2019-12-08T15:06:40.803 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T15:06:41.016 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-12-08T15:06:41.027 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-12-08T15:06:41.673 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting botocore<1.14.0,>=1.13.34 2019-12-08T15:06:41.683 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/59/cb/b4772a4abc128f6fd637af9d7fc93a1db11617859af680fc2b2e6282eb95/botocore-1.13.34-py2.py3-none-any.whl (5.8MB) 2019-12-08T15:06:42.197 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-12-08T15:06:42.207 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-12-08T15:06:42.397 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-08T15:06:42.399 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T15:06:42.461 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting six 2019-12-08T15:06:42.472 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-08T15:06:42.561 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-12-08T15:06:42.572 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-12-08T15:06:42.667 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting idna<2.9,>=2.5 2019-12-08T15:06:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-12-08T15:06:42.796 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-12-08T15:06:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-12-08T15:06:42.914 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting certifi>=2017.4.17 2019-12-08T15:06:42.923 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB) 2019-12-08T15:06:43.048 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting docutils<0.16,>=0.10 2019-12-08T15:06:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-12-08T15:06:43.254 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-12-08T15:06:43.264 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-12-08T15:06:43.408 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-12-08T15:06:43.418 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-12-08T15:06:43.460 INFO:teuthology.orchestra.run.smithi129.stdout:Building wheels for collected packages: ordereddict 2019-12-08T15:06:43.461 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for ordereddict (setup.py): started 2019-12-08T15:06:43.709 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-12-08T15:06:43.710 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=b6297f229daa37756d8acbba17adc749b06304fb7352f2fd28e0c72bb1534472 2019-12-08T15:06:43.710 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-12-08T15:06:43.712 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully built ordereddict 2019-12-08T15:06:43.792 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-12-08T15:06:45.987 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 boto3-1.10.34 botocore-1.13.34 bunch-1.0.1 certifi-2019.11.28 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.2 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-12-08T15:06:46.285 INFO:teuthology.orchestra.run.smithi129.stdout:running develop 2019-12-08T15:06:46.345 INFO:teuthology.orchestra.run.smithi129.stdout:running egg_info 2019-12-08T15:06:46.345 INFO:teuthology.orchestra.run.smithi129.stdout:creating s3tests.egg-info 2019-12-08T15:06:46.345 INFO:teuthology.orchestra.run.smithi129.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-12-08T15:06:46.345 INFO:teuthology.orchestra.run.smithi129.stdout:writing s3tests.egg-info/PKG-INFO 2019-12-08T15:06:46.346 INFO:teuthology.orchestra.run.smithi129.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-12-08T15:06:46.346 INFO:teuthology.orchestra.run.smithi129.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-12-08T15:06:46.347 INFO:teuthology.orchestra.run.smithi129.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-12-08T15:06:46.347 INFO:teuthology.orchestra.run.smithi129.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-08T15:06:46.348 INFO:teuthology.orchestra.run.smithi129.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-08T15:06:46.349 INFO:teuthology.orchestra.run.smithi129.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-08T15:06:46.350 INFO:teuthology.orchestra.run.smithi129.stdout:running build_ext 2019-12-08T15:06:46.350 INFO:teuthology.orchestra.run.smithi129.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-12-08T15:06:46.350 INFO:teuthology.orchestra.run.smithi129.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-12-08T15:06:46.353 INFO:teuthology.orchestra.run.smithi129.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-08T15:06:46.353 INFO:teuthology.orchestra.run.smithi129.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-08T15:06:46.353 INFO:teuthology.orchestra.run.smithi129.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-08T15:06:46.353 INFO:teuthology.orchestra.run.smithi129.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-08T15:06:46.354 INFO:teuthology.orchestra.run.smithi129.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-08T15:06:46.354 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.354 INFO:teuthology.orchestra.run.smithi129.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-12-08T15:06:46.354 INFO:teuthology.orchestra.run.smithi129.stdout:Processing dependencies for s3tests==0.0.1 2019-12-08T15:06:46.404 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for isodate==0.6.0 2019-12-08T15:06:46.404 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: isodate 0.6.0 2019-12-08T15:06:46.404 INFO:teuthology.orchestra.run.smithi129.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-08T15:06:46.404 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.405 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.405 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for gevent==1.4.0 2019-12-08T15:06:46.405 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: gevent 1.4.0 2019-12-08T15:06:46.406 INFO:teuthology.orchestra.run.smithi129.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-08T15:06:46.406 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.406 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.407 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for bunch==1.0.1 2019-12-08T15:06:46.407 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: bunch 1.0.1 2019-12-08T15:06:46.407 INFO:teuthology.orchestra.run.smithi129.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-08T15:06:46.407 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.407 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.408 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for PyYAML==5.2 2019-12-08T15:06:46.408 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: PyYAML 5.2 2019-12-08T15:06:46.408 INFO:teuthology.orchestra.run.smithi129.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-08T15:06:46.408 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.409 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.409 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for boto3==1.10.34 2019-12-08T15:06:46.410 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: boto3 1.10.34 2019-12-08T15:06:46.410 INFO:teuthology.orchestra.run.smithi129.stdout:Adding boto3 1.10.34 to easy-install.pth file 2019-12-08T15:06:46.410 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.410 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.410 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for boto==2.49.0 2019-12-08T15:06:46.410 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: boto 2.49.0 2019-12-08T15:06:46.412 INFO:teuthology.orchestra.run.smithi129.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-08T15:06:46.412 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.412 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.413 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for six==1.13.0 2019-12-08T15:06:46.413 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: six 1.13.0 2019-12-08T15:06:46.413 INFO:teuthology.orchestra.run.smithi129.stdout:Adding six 1.13.0 to easy-install.pth file 2019-12-08T15:06:46.413 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.413 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.413 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for greenlet==0.4.15 2019-12-08T15:06:46.415 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: greenlet 0.4.15 2019-12-08T15:06:46.416 INFO:teuthology.orchestra.run.smithi129.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-08T15:06:46.416 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.416 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.416 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for s3transfer==0.2.1 2019-12-08T15:06:46.416 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: s3transfer 0.2.1 2019-12-08T15:06:46.417 INFO:teuthology.orchestra.run.smithi129.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-12-08T15:06:46.417 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.417 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.417 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for botocore==1.13.34 2019-12-08T15:06:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: botocore 1.13.34 2019-12-08T15:06:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Adding botocore 1.13.34 to easy-install.pth file 2019-12-08T15:06:46.418 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for jmespath==0.9.4 2019-12-08T15:06:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: jmespath 0.9.4 2019-12-08T15:06:46.419 INFO:teuthology.orchestra.run.smithi129.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-12-08T15:06:46.419 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.419 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.419 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for futures==3.3.0 2019-12-08T15:06:46.419 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: futures 3.3.0 2019-12-08T15:06:46.420 INFO:teuthology.orchestra.run.smithi129.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-12-08T15:06:46.421 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.421 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.421 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for urllib3==1.25.7 2019-12-08T15:06:46.421 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: urllib3 1.25.7 2019-12-08T15:06:46.421 INFO:teuthology.orchestra.run.smithi129.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-12-08T15:06:46.421 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.422 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.422 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for python-dateutil==2.8.0 2019-12-08T15:06:46.422 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: python-dateutil 2.8.0 2019-12-08T15:06:46.422 INFO:teuthology.orchestra.run.smithi129.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-12-08T15:06:46.422 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.422 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.423 INFO:teuthology.orchestra.run.smithi129.stdout:Searching for docutils==0.15.2 2019-12-08T15:06:46.423 INFO:teuthology.orchestra.run.smithi129.stdout:Best match: docutils 0.15.2 2019-12-08T15:06:46.423 INFO:teuthology.orchestra.run.smithi129.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-12-08T15:06:46.424 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T15:06:46.424 INFO:teuthology.orchestra.run.smithi129.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-08T15:06:46.424 INFO:teuthology.orchestra.run.smithi129.stdout:Finished processing dependencies for s3tests==0.0.1 2019-12-08T15:06:46.429 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-12-08T15:06:46.476 INFO:tasks.s3tests:Configuring boto... 2019-12-08T15:06:46.477 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-08T15:06:46.565 INFO:teuthology.orchestra.run.smithi129:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-12-08T15:06:47.809 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:23:19.235 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:47.849 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:23:19.275 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:47.893 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:23:19.317 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:47.897 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:23:19.321 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:47.927 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:23:19.518 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:47.933 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:23:19.524 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:47.938 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:23:19.364 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:48.014 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:23:19.438 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:48.082 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:23:19.673 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:06:51.758 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-12-08T15:06:54.816 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-12-08T15:06:58.857 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-12-08T15:07:02.466 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-12-08T15:07:04.912 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-12-08T15:07:07.610 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-12-08T15:07:08.112 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:23:39.538 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.129 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:23:39.555 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.137 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:23:39.561 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.157 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:23:39.581 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.161 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:23:39.587 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.183 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:23:39.607 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.201 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:23:39.792 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.211 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:23:39.802 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:08.266 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:23:39.857 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:07:10.671 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-12-08T15:07:12.748 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-12-08T15:07:12.939 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-12-08T15:07:13.728 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-12-08T15:07:16.148 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-12-08T15:07:19.409 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-12-08T15:07:22.034 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-12-08T15:07:24.434 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-12-08T15:07:24.765 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-12-08T15:07:24.842 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-12-08T15:07:24.842 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-12-08T15:07:24.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-12-08T15:07:24.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-12-08T15:07:24.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-08T15:07:24.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-12-08T15:07:24.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-12-08T15:07:24.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-12-08T15:07:24.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-08T15:07:24.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-12-08T15:07:24.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-08T15:07:24.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-12-08T15:07:24.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-12-08T15:07:24.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-12-08T15:07:24.845 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-12-08T15:07:24.846 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-12-08T15:07:24.846 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-12-08T15:07:24.848 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-08T15:07:24.848 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-12-08T15:07:24.848 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-12-08T15:07:24.848 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-12-08T15:07:24.848 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-08T15:07:24.849 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-12-08T15:07:24.849 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-08T15:07:24.849 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-12-08T15:07:24.849 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-12-08T15:07:24.849 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-12-08T15:07:24.850 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-08T15:07:24.851 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-12-08T15:07:24.851 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-08T15:07:24.851 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-08T15:07:24.851 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-08T15:07:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-12-08T15:07:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-12-08T15:07:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-12-08T15:07:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-12-08T15:07:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-12-08T15:07:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-12-08T15:07:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-12-08T15:07:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-12-08T15:07:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-12-08T15:07:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-12-08T15:07:47.656 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-12-08T15:07:56.668 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-12-08T15:09:48.741 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:26:20.167 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.746 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:26:20.171 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.759 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:26:20.183 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.761 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:26:20.185 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.770 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:26:20.361 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.775 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:26:20.366 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.870 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:26:20.293 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.880 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:26:20.305 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:09:48.951 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:26:20.541 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:10:07.274 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-12-08T15:13:10.716 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-12-08T15:13:20.146 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-12-08T15:13:24.230 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-12-08T15:13:28.676 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-12-08T15:13:33.190 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-12-08T15:13:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-12-08T15:13:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-12-08T15:13:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-12-08T15:13:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-12-08T15:13:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-12-08T15:13:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-12-08T15:13:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-12-08T15:13:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-12-08T15:13:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-12-08T15:13:33.198 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-08T15:13:33.198 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-08T15:13:33.198 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-12-08T15:13:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-12-08T15:13:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-08T15:13:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-08T15:13:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-12-08T15:13:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-12-08T15:13:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-08T15:13:33.200 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-08T15:13:33.200 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-12-08T15:13:33.200 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-12-08T15:13:33.200 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-12-08T15:13:33.200 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-12-08T15:13:33.200 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-12-08T15:13:33.201 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-12-08T15:13:33.201 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-12-08T15:13:33.201 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-12-08T15:13:33.201 INFO:teuthology.orchestra.run.smithi129.stderr:SKIP 2019-12-08T15:13:33.205 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-12-08T15:14:42.876 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-08T15:14:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-08T15:14:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-08T15:14:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-08T15:14:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-08T15:14:42.877 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-08T15:14:42.878 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-08T15:14:43.244 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-08T15:14:43.409 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-08T15:14:43.741 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-08T15:14:43.764 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-08T15:14:43.784 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-08T15:14:43.784 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-08T15:14:43.784 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-08T15:14:43.784 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-08T15:14:43.784 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-08T15:14:43.785 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-08T15:14:43.785 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-08T15:14:43.785 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-08T15:14:43.785 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-08T15:14:43.786 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-08T15:14:43.786 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-08T15:14:43.789 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-08T15:14:43.789 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-08T15:14:43.790 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-08T15:14:43.790 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-08T15:14:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-08T15:14:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-08T15:14:43.794 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-08T15:14:43.795 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-08T15:14:43.819 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-08T15:14:43.843 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-08T15:14:43.843 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-08T15:14:43.991 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-08T15:14:44.755 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:44.756 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:14:45.664 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:45.664 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:14:46.340 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:46.340 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:14:47.049 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:47.050 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:14:49.723 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:49.723 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:14:49.923 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:14:50.434 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:50.434 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:14:52.913 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:14:52.913 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:00.871 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:31:32.297 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:00.878 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:31:32.304 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:00.887 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:31:32.310 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:00.889 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:31:32.313 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:00.892 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:31:32.481 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:00.906 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:31:32.496 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:01.064 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:31:32.487 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:01.100 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:31:32.525 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:01.229 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:31:32.819 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:02.159 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:02.490 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:02.491 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:05.637 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:05.637 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:05.774 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:06.261 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:06.261 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:08.322 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:08.322 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:09.521 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:31:40.946 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.540 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:31:40.966 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.561 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:31:40.984 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.580 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:31:41.005 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.597 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:31:41.021 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.598 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:31:41.188 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.600 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:31:41.190 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.614 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:31:41.038 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.628 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:31:41.218 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:15:09.737 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:09.737 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:10.943 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:11.206 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:11.206 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:14.139 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:14.139 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:15.869 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:15.869 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:18.005 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:18.368 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:18.369 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:21.810 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:21.811 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:24.045 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:24.045 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:25.875 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:28.751 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:28.752 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:30.800 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:30.801 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:34.203 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:34.539 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:34.540 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:36.696 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:36.696 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:36.799 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:37.023 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:37.023 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:39.121 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:39.121 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:40.160 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:40.160 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:42.026 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:42.303 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:42.303 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:44.815 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:44.815 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:46.079 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:46.079 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:48.196 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:48.197 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:49.843 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:50.061 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:50.062 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:52.479 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:52.480 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:52.698 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:52.946 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:52.946 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:54.780 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:55.055 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:55.055 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:57.595 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:57.903 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:57.903 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:57.992 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:15:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:58.269 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:15:59.990 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:15:59.991 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:00.082 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:00.397 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:00.397 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:02.497 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:02.497 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:05.562 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:05.818 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:05.818 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:08.506 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:08.506 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:11.093 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:11.093 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:13.367 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:13.368 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:13.464 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:13.701 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:13.701 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:16.629 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:16.629 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:16.702 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:16.970 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:20.948 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:20.949 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:21.021 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:21.228 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:21.228 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:23.864 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:23.864 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:24.057 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:24.425 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:24.426 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:26.947 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:26.948 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:27.159 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:27.472 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:27.473 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:30.403 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:30.404 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:33.280 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:33.280 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:33.674 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:33.675 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:33.786 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:34.006 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:34.006 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:34.131 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:34.636 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:34.637 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:34.767 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:35.034 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:35.034 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:35.131 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:35.618 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:35.619 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:38.633 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:38.936 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:38.937 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:40.994 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:40.995 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:44.701 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:44.954 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:44.955 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:50.585 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:50.585 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:51.685 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:16:52.374 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:52.374 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:16:58.549 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:16:59.789 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:17:00.110 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:00.110 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:07.363 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:17:08.151 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:08.152 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:15.736 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:15.736 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:16.310 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:17:16.620 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:16.620 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:24.492 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:24.492 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:24.719 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:17:25.089 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:25.090 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:35.546 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:35.547 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:36.519 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:36.519 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:37.122 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:37.122 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:37.975 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:37.975 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:38.908 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:38.909 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:39.703 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:39.703 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:40.871 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:42.359 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:42.359 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:42.695 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:17:43.078 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:43.078 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:52.141 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:52.141 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:53.146 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:53.146 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:54.010 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:54.010 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:54.923 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:54.923 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:55.542 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:55.630 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:56.797 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:56.797 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:57.320 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:57.320 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:58.423 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:58.423 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:17:59.045 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:17:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:17:59.359 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:04.140 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:04.140 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:04.674 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:18:04.942 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:04.942 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:08.546 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:08.547 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:08.843 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:18:09.089 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:09.089 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:17.814 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:17.814 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:18.325 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:18:18.674 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:18.675 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:26.302 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:26.303 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:26.808 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:18:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:27.076 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:37.711 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:37.711 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:38.737 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:38.737 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:40.193 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:40.193 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:40.737 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:40.737 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:41.244 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:41.244 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:41.872 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:41.872 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:42.630 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:42.630 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:43.566 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:18:43.867 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:43.867 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:53.943 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:53.943 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:54.938 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:54.938 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:55.511 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:55.511 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:56.511 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:56.511 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:57.346 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:57.347 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:58.819 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:18:59.607 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:18:59.607 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:00.859 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:00.859 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:01.259 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:01.472 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:01.472 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:10.107 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:10.107 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:10.731 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:11.215 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:11.215 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:18.561 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:18.839 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:19.121 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:19.122 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:27.429 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:27.429 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:28.043 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:28.439 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:28.439 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:34.904 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:34.904 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:35.288 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:35.535 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:35.536 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:44.564 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:44.564 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:44.819 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:45.064 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:45.064 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:53.097 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:53.098 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:19:54.026 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:19:54.320 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:19:54.320 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:02.583 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:02.583 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:02.973 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:03.289 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:03.290 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:11.029 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:11.029 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:11.336 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:11.546 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:36:42.971 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.549 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:36:42.974 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.558 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:36:42.981 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.563 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:36:42.986 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.572 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:36:43.161 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.580 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:36:43.169 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:11.670 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:11.748 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:36:43.171 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.749 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:36:43.174 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:11.817 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:36:43.406 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:20:18.285 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:18.285 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:19.182 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:19.450 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:19.450 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:26.644 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:27.103 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:27.441 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:27.441 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:35.442 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:35.442 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:36.204 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:36.752 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:36.752 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:45.944 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:45.944 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:46.923 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:47.182 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:47.182 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:54.931 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:54.931 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:20:55.482 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:20:55.727 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:20:55.728 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:02.012 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:02.013 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:02.568 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:21:02.911 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:02.911 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:08.523 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:08.523 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:09.069 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:21:09.703 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:09.703 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:19.307 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:19.308 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:20.371 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:21:20.618 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:20.619 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:27.891 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:27.891 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:21:28.372 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:21:28.948 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:21:28.948 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:23:31.062 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:40:02.486 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.066 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:40:02.491 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.105 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:40:02.528 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.117 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:40:02.534 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.154 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:40:02.742 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.160 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:40:02.585 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.161 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:40:02.749 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.197 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:40:02.620 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:23:31.263 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:40:02.852 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.178 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:41:54.602 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.192 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:41:54.614 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.192 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:41:54.616 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.198 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:41:54.786 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.198 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:41:54.621 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.203 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:41:54.791 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.272 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:41:54.697 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.318 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:41:54.740 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:25:23.327 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:41:54.915 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:35.854 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:47:07.279 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:35.859 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:47:07.284 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:35.871 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:47:07.294 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:35.876 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:47:07.299 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:35.876 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:47:07.463 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:35.881 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:47:07.468 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:36.067 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:47:07.492 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:36.082 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:47:07.669 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:30:36.083 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:47:07.506 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.518 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:48:23.943 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.519 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:48:23.944 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.561 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:48:23.984 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.567 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:48:23.992 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.567 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:48:23.991 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.573 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:48:24.160 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.581 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:48:24.168 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.617 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:48:24.041 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:31:52.631 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:48:24.218 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.452 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:52:18.877 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.461 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:52:18.887 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.466 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:52:18.890 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.472 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:52:19.058 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.472 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:52:18.893 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.477 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:52:19.064 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.674 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:52:19.261 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.687 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:52:19.111 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:35:47.692 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:52:19.118 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:13.968 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:56:45.394 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:13.971 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:56:45.397 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.033 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:56:45.457 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.036 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:56:45.460 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.060 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:56:45.648 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.067 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:56:45.654 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.116 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:56:45.542 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.200 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:56:45.624 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:14.260 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:56:45.848 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.188 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:57:26.614 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.198 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:57:26.624 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.206 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:57:26.630 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.210 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:57:26.634 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.210 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:57:26.798 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.220 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:57:26.808 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.259 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:02:57:26.683 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.263 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:02:57:26.689 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:40:55.299 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:02:57:26.886 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:44:14.758 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:14.759 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:16.820 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:16.820 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:22.497 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:22.497 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:23.062 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:23.681 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:23.681 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:29.856 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:29.857 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:30.521 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:30.919 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:30.920 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:36.340 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:36.340 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:37.499 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:38.108 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:38.108 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:44.876 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:44.876 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:45.249 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:45.876 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:45.876 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:52.090 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:52.091 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:52.411 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:52.667 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:52.667 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:58.662 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:58.662 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:44:59.549 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:44:59.807 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:44:59.807 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:05.411 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:05.411 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:05.895 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:06.121 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:06.121 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:13.237 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:13.237 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:14.096 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:14.484 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:14.484 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:20.661 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:20.661 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:21.130 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:21.407 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:21.407 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:27.729 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:27.729 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:28.058 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:28.409 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:28.409 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:33.951 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:33.951 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:34.601 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:34.901 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:34.901 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:38.175 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:38.431 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:38.652 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:38.653 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:44.094 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:44.094 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:44.479 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:44.718 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:44.718 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:51.687 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:51.687 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:52.600 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:53.086 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:53.086 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:58.996 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:58.996 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:45:59.410 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:45:59.797 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:45:59.798 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:06.043 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:06.043 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:06.044 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:02:37.469 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.049 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:02:37.476 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.053 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:02:37.477 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.058 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:02:37.482 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.066 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:02:37.654 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.075 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:02:37.664 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.257 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:02:37.683 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.277 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:02:37.701 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.320 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:06.375 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:02:37.961 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:46:06.575 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:06.575 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:13.016 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:13.016 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:13.677 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:14.280 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:14.280 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:21.805 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:21.805 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:22.306 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:22.848 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:22.848 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:30.404 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:30.404 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:31.307 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:31.637 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:31.638 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:38.251 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:38.251 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:38.733 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:38.993 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:38.993 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:44.599 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:44.599 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:44.985 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:45.461 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:45.462 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:52.187 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:52.656 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:46:52.914 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:52.915 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:58.744 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:46:58.744 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:46:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:47:00.142 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:00.142 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:05.835 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:05.835 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:07.242 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:47:07.485 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:07.485 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:14.037 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:14.037 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:47:14.530 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:14.530 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:21.896 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:21.896 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:23.006 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:47:23.255 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:23.255 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:31.247 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:31.247 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:31.595 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:47:31.957 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:31.957 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:39.739 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:40.405 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:47:40.834 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:47:40.834 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:47:55.446 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:04:26.872 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.489 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:04:26.915 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.508 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:04:26.932 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.511 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:04:26.936 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.555 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:04:27.143 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.564 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:04:27.152 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.587 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:04:27.011 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.628 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:04:27.053 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:47:55.733 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:04:27.321 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:13.857 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:48:13.857 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:48:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:48:17.057 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:48:17.057 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:48:35.738 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:05:07.164 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.753 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:05:07.179 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.768 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:05:07.192 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.784 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:05:07.209 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.817 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:05:07.241 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.820 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:05:07.246 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.826 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:05:07.414 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.836 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:05:07.424 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:35.943 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:05:07.531 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:48:55.044 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:48:55.044 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:48:57.784 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:48:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:48:58.299 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:05.447 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:05.448 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:05.583 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:49:06.008 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:06.008 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:14.988 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:14.988 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:15.762 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:22.991 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:22.991 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:23.440 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:49:23.751 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:23.752 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:32.740 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:33.336 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:49:33.642 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:33.642 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:42.330 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:42.330 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:42.749 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:49:43.038 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:43.039 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:53.382 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:53.382 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:49:53.818 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:49:54.118 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:49:54.118 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:05.056 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:05.056 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:50:06.161 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:06.161 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:14.212 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:14.212 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:14.526 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:50:15.011 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:15.011 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:22.331 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:22.331 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:23.044 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:50:23.269 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:23.269 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:29.955 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:29.955 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:30.642 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:50:31.177 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:31.178 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:37.064 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:37.064 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:40.690 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:50:40.942 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:40.942 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:43.333 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:43.334 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:44.709 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:44.709 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:50:58.123 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:50:58.123 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:04.535 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:04.536 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:06.822 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:06.822 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:08.646 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:08.646 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:09.925 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:12.917 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:12.917 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:14.823 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:16.226 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:07:47.652 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.237 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:07:47.663 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.243 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:07:47.667 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.247 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:07:47.672 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.253 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:07:47.842 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.261 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:07:47.850 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.357 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:07:47.783 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.365 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:07:47.789 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:16.436 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:07:48.025 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:51:17.420 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:17.420 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:17.942 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:18.432 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:18.432 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:21.402 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:21.402 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:21.702 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:21.935 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:21.935 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:22.239 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:22.566 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:22.566 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:22.916 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:23.289 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:23.289 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:23.743 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:24.267 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:24.268 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:29.274 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:29.274 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:30.319 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:30.628 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:30.628 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:33.321 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:33.613 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:33.613 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:33.886 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:34.071 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:34.072 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:34.255 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:34.255 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:35.466 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:35.466 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:35.478 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:35.478 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:35.881 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:35.881 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:36.285 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:36.285 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:36.564 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:36.564 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:36.966 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:36.967 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:38.187 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:38.187 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:38.424 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:38.424 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:38.604 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:38.604 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:39.105 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:39.105 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:39.356 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:40.083 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:40.083 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:40.501 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:40.501 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:40.813 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:40.813 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:41.014 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:41.014 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:42.356 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:42.830 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:43.485 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:43.486 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:43.907 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:43.907 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:44.105 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:44.106 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:44.556 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:44.557 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:45.368 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:45.369 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:45.561 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:45.562 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:45.776 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:46.531 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:46.532 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:47.130 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:47.131 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:47.138 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:47.138 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:47.834 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:47.834 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:48.087 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:48.087 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:48.315 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:48.315 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:48.609 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:48.609 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:49.047 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:49.688 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:49.688 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:49.939 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:49.939 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:51.335 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:51.840 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:51.840 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:52.098 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:52.099 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:52.513 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:52.513 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:52.828 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:52.828 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:53.013 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:53.014 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:53.708 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:53.709 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:54.191 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:54.192 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:54.476 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:54.477 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:54.727 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:54.728 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:55.365 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:55.365 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:51:59.255 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:51:59.256 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:51:59.808 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:00.041 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:00.042 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:02.038 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:02.038 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:06.532 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:07.301 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:07.302 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:09.648 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:09.648 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:09.852 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:10.283 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:12.939 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:12.939 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:13.557 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:13.985 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:13.985 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:18.506 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:18.507 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:26.913 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:27.218 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:27.218 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:32.601 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:32.602 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:38.985 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:39.674 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:39.674 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:42.555 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:42.555 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:43.997 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:44.568 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:44.568 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:46.822 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:46.822 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:47.705 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:47.998 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:47.999 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:49.942 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:49.943 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:51.495 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:51.800 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:51.800 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:55.189 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:55.189 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:57.300 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:52:57.567 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:57.567 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:52:59.881 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:52:59.881 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:05.696 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:06.003 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:06.003 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:08.624 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:08.624 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:10.427 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:10.686 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:10.687 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:13.971 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:13.971 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:16.125 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:16.610 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:18.793 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:18.793 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:20.655 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:20.655 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:22.185 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:22.625 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:22.625 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:26.113 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:26.113 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:28.435 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:28.436 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:30.897 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:30.897 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:34.633 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:34.884 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:34.885 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:37.334 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:37.334 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:40.011 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:40.443 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:40.443 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:42.421 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:42.421 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:44.572 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:44.953 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:44.953 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:48.349 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:48.349 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:50.613 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:50.613 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:51.090 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:51.499 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:51.499 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:53.537 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:53.538 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:54.547 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:54.820 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:54.820 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:57.446 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:57.446 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:53:58.082 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:53:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:53:58.702 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:00.976 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:00.976 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:01.797 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:02.023 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:02.023 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:04.739 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:04.739 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:06.029 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:06.259 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:06.259 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:08.072 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:08.072 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:14.742 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:14.743 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:18.649 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:19.060 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:19.060 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:22.034 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:22.034 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:23.782 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:23.783 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:24.396 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:24.927 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:24.928 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:28.061 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:28.061 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:30.110 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:30.110 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:32.850 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:32.851 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:34.138 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:34.425 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:34.426 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:36.951 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:36.951 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:38.572 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:38.572 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:39.052 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:39.435 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:39.435 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:41.578 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:41.578 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:43.922 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:44.162 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:44.162 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:46.850 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:46.850 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:47.858 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:48.276 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:48.276 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:50.903 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:50.904 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:51.321 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:52.046 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:52.046 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:54.685 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:54.685 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:55.115 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:55.337 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:55.338 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:57.475 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:57.475 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:54:57.738 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:54:58.052 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:54:58.052 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:00.919 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:00.919 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:01.270 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:01.632 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:01.632 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:04.432 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:04.432 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:05.008 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:05.259 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:05.260 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:09.210 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:09.211 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:11.157 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:11.384 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:11.759 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:13.663 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:13.663 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:14.140 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:14.389 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:14.389 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:17.031 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:17.032 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:17.879 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:18.155 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:18.155 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:20.360 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:20.361 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:21.074 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:21.389 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:21.389 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:24.518 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:25.096 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:25.350 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:25.351 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:28.888 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:28.888 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:29.315 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:29.569 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:29.570 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:32.189 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:32.189 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:32.838 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:33.083 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:33.083 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:37.089 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:37.089 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:37.440 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:37.669 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:37.669 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:40.240 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:40.240 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:41.305 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:41.547 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:41.547 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:43.595 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:43.595 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:43.912 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:44.171 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:44.171 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:46.373 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:46.373 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:46.993 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:46.993 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:49.381 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:49.381 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:49.730 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:50.050 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:50.050 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:52.276 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:52.277 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:52.537 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:53.114 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:53.114 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:55.632 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:55.633 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:57.137 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:55:57.341 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:57.341 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:55:59.922 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:55:59.923 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:01.482 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:01.696 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:01.697 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:03.086 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:03.086 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:04.581 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:07.240 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:07.240 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:09.327 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:09.752 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:09.752 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:12.150 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:12.150 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:14.030 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:14.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:14.287 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:16.774 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:16.775 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:19.092 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:19.092 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:19.338 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:19.586 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:19.586 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:23.253 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:23.253 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:25.009 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:25.492 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:25.492 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:27.678 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:27.678 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:28.115 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:12:59.541 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.118 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:12:59.544 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.129 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:12:59.719 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.133 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:12:59.723 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.154 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:12:59.578 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.156 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:12:59.580 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.346 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:12:59.772 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.351 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:12:59.941 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.370 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:12:59.794 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:29.074 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:29.074 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:31.080 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:31.080 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:33.525 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:33.889 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:35.794 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:35.794 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:38.150 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:38.425 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:38.425 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:41.348 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:41.348 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:44.256 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:44.512 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:44.512 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:46.714 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:46.714 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:47.717 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:48.088 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:48.088 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:50.058 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:50.058 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:53.528 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:54.123 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:54.124 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:55.952 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:55.952 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:56:57.206 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:13:28.625 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.239 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:13:28.665 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.241 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:13:28.665 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.243 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:13:28.667 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.285 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:13:28.707 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.285 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:13:28.709 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.286 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:13:28.876 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.291 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:13:28.881 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:57.329 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:13:28.919 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T15:56:58.033 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:56:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:56:58.263 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:00.429 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:00.429 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:01.372 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:02.022 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:02.022 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:04.078 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:04.078 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:06.931 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:07.165 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:07.166 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:10.865 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:10.865 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:11.118 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:11.393 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:11.393 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:15.331 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:15.331 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:21.205 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:21.205 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:21.679 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:21.920 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:21.920 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:26.038 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:26.038 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:32.339 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:32.340 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:33.082 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:33.793 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:33.793 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:37.431 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:37.431 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:39.394 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:39.395 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:39.915 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:40.234 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:40.234 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:43.937 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:43.938 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:44.357 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:44.667 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:44.667 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:47.416 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:47.417 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:48.156 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:48.156 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:57:55.625 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:57:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:57:55.885 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:00.290 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:00.290 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:00.444 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:00.680 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:00.680 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:04.765 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:04.765 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:06.136 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:06.369 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:06.370 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:09.763 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:09.764 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:09.976 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:10.201 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:10.201 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:14.733 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:14.733 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:15.227 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:15.663 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:15.663 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:20.311 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:20.311 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:20.446 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:20.715 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:20.716 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:24.998 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:24.998 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:25.259 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:25.567 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:25.568 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:29.525 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:29.526 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:37.046 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:37.634 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:37.634 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:41.434 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:41.434 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:43.117 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:43.390 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:43.390 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:47.316 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:47.317 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:47.462 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:47.710 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:47.710 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:52.073 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:52.073 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:52.147 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:52.350 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:52.350 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:55.983 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:55.983 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:56.130 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:58:56.623 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:56.623 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:59.716 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:58:59.716 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:58:59.787 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:00.480 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:00.480 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:03.775 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:05.073 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:05.073 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:05.903 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:06.306 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:09.263 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:09.264 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:10.977 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:10.977 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:12.818 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:13.263 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:13.263 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:16.003 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:16.003 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:17.334 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:17.623 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:17.624 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:20.445 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:20.446 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:21.896 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:21.896 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:21.967 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:22.172 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:22.172 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:22.628 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:23.130 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:23.130 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:23.544 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:23.958 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:23.959 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:24.210 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:24.635 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:24.635 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:25.418 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:25.674 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:25.674 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:28.803 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:29.123 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:29.124 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:32.449 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:32.729 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:32.729 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:36.002 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:36.003 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:37.592 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:37.880 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:37.880 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:40.908 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:40.908 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:41.516 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:41.751 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:41.751 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:44.133 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:44.631 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:49.223 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:49.597 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:49.597 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:51.982 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:52.187 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:54.706 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:55.039 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:55.040 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:56.888 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:56.889 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T15:59:58.073 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T15:59:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T15:59:58.289 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:00.891 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:00.891 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:02.154 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:02.419 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:02.420 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:04.512 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:04.512 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:08.321 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:08.602 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:08.603 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:11.476 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:11.476 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:13.446 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:13.756 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:13.756 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:16.363 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:16.364 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:17.759 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:18.485 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:18.485 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:21.723 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:21.723 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:26.906 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:27.141 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:29.240 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:33.161 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:33.474 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:33.474 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:37.994 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:37.995 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:40.057 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:40.308 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:40.309 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:45.292 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:45.292 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:45.865 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:47.862 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:48.121 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:48.122 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:49.833 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:49.833 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:50.866 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:51.501 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:51.502 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:54.093 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:54.094 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:00:57.443 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:00:57.677 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:00:57.677 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:00.180 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:00.180 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:02.780 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:03.050 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:03.050 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:05.379 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:05.379 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:09.842 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:09.842 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:12.617 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:12.617 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:16.074 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:16.314 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:16.314 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:19.085 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:19.086 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:22.375 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:22.581 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:22.582 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:25.358 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:25.358 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:26.007 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:26.007 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:28.216 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:28.216 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:29.006 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:29.007 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:29.741 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:29.742 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:31.459 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:31.459 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:32.727 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:32.727 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:33.004 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:33.281 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:36.094 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:36.094 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:36.565 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:38.136 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:38.136 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:38.797 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:38.797 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:39.028 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:18:10.454 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.034 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:18:10.460 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.048 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:18:10.472 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.053 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:18:10.477 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.061 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:18:10.650 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.078 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:18:10.668 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.172 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:39.172 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:39.198 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:18:10.624 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.218 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:18:10.642 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.354 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:18:10.944 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:01:39.543 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:39.543 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:39.654 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:39.889 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:39.889 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:41.625 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:41.626 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:42.110 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:42.110 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:43.584 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:43.584 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:44.171 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:44.171 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:44.513 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:44.514 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:44.999 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:45.000 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:45.282 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:45.544 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:45.544 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:48.349 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:48.350 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:48.972 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:48.972 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:51.715 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:52.086 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:52.087 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:53.321 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:53.321 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:53.498 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:01:53.737 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:53.737 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:56.597 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:56.597 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:57.513 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:57.514 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:59.012 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:59.012 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:01:59.334 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:01:59.335 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:00.113 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:00.114 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:00.557 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:01.382 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:01.686 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:01.686 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:04.194 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:04.194 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:04.707 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:04.707 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:05.234 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:05.489 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:08.012 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:08.013 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:11.695 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:11.695 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:12.206 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:12.206 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:15.747 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:15.986 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:15.987 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:18.676 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:18.676 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:19.579 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:19.794 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:23.132 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:23.133 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:25.189 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:25.443 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:25.443 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:27.490 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:27.490 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:27.948 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:27.948 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:30.788 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:31.583 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:31.583 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:34.272 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:34.273 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:35.111 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:35.111 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:35.698 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:36.178 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:36.179 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:38.860 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:38.861 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:43.983 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:44.279 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:44.279 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:47.712 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:47.712 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:52.646 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:52.646 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:54.368 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:54.803 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:54.804 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:54.962 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:02:55.269 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:55.269 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:02:57.892 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:02:57.892 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:02.434 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:02.435 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:03.902 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:04.295 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:04.416 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:03:04.830 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:04.831 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:06.401 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:06.402 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:10.441 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:10.441 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:11.593 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:11.593 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:12.514 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:12.514 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:13.008 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:03:13.218 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:13.218 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:15.573 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:15.573 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:21.309 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:21.310 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:22.680 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:22.681 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:23.609 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:23.610 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:24.255 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:03:24.492 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:24.492 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:26.713 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:26.713 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:30.481 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:30.481 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:31.601 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:31.602 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:33.123 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:33.123 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:33.588 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:03:33.927 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:33.927 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:37.530 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:37.530 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:42.147 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:42.147 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:43.149 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:43.149 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:44.103 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:44.103 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:44.382 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:03:44.667 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:44.667 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:47.057 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:47.057 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:52.264 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:52.265 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:53.541 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:53.541 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:54.953 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:54.954 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:55.651 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:03:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:56.100 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:03:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:03:57.962 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:02.072 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:02.072 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:02.754 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:02.755 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:03.437 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:03.438 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:04.735 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:04:05.092 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:05.092 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:08.257 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:08.257 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:14.226 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:14.226 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:15.417 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:15.417 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:16.332 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:16.332 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:17.448 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:04:17.676 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:17.676 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:20.088 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:20.089 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:24.573 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:24.573 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:31.586 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:04:31.794 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:31.795 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:33.688 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:33.688 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:38.364 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:38.364 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:45.239 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:04:45.688 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:45.688 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:48.897 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:48.898 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:04:55.212 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:04:55.212 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:01.052 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:01.052 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:16.248 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:16.248 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:17.912 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:18.300 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:18.300 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:18.681 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:21:50.107 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.688 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:21:50.114 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.699 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:21:50.123 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.702 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:21:50.125 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.734 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:21:50.323 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.743 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:21:50.333 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.789 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:21:50.214 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.805 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:21:50.229 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.876 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:21:50.466 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:05:18.879 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:19.110 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:19.110 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:19.909 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:20.191 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:20.191 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:20.345 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:20.589 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:20.589 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:22.213 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:22.213 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:24.821 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:24.821 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:31.401 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:31.678 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:31.678 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:32.853 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:32.854 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:34.813 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:34.813 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:38.580 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:38.843 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:38.844 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:40.173 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:40.584 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:40.584 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:43.075 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:43.075 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:47.509 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:47.509 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:48.241 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:48.241 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:50.262 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:50.262 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:05:51.552 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:05:51.872 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:05:51.872 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:07.880 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:07.880 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:08.387 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:08.387 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:24.301 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:06:24.579 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:24.579 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:33.153 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:33.153 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:33.841 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:33.841 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:34.149 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:06:34.405 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:34.405 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:38.902 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:38.903 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:42.623 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:06:43.011 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:43.011 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:45.268 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:45.269 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:50.347 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:23:21.773 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.353 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:06:50.394 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:23:21.820 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.401 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:23:21.825 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.408 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:23:21.833 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.429 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:23:22.017 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.438 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:23:22.024 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.464 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:23:21.890 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.472 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:23:21.897 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.520 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:23:22.110 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:06:50.686 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:50.686 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:52.957 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:52.957 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:56.257 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:06:56.483 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:56.483 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:58.391 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:58.392 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:06:59.608 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:06:59.849 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:06:59.849 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:02.379 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:02.380 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:05.670 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:07:05.939 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:05.939 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:08.948 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:08.948 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:12.101 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:12.101 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:16.525 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:07:16.787 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:16.787 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:19.595 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:19.595 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:23.348 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:07:23.646 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:23.647 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:28.112 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:28.112 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:29.592 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:29.592 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:31.747 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:31.747 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:32.473 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:32.473 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:34.849 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:07:35.160 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:35.160 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:38.419 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:38.419 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:40.940 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:40.941 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:43.542 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:43.542 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:43.803 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:07:44.067 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:44.068 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:45.920 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:45.921 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:50.151 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:50.151 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:54.098 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:54.098 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:07:58.109 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:07:58.264 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:07:58.226+0000 5deb3700 -1 handle_data failed with exit code -32 2019-12-08T16:07:58.470 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:07:58.471 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:00.567 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:04.320 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:06.583 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:06.583 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:10.745 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:08:10.801 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:08:10.780+0000 92f1d700 -1 handle_data failed with exit code -32 2019-12-08T16:08:11.007 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:11.008 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:13.739 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:13.740 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:14.262 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:08:14.484 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:14.484 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:16.482 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:16.482 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:08:17.252 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:17.253 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:19.364 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:21.044 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:21.044 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:26.412 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:26.413 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:35.933 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:35.934 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:38.981 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:38.981 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:46.814 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:46.814 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:56.307 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:08:56.630 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:56.630 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:08:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:08:58.698 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:09:07.521 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:09:07.737 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:09:07.737 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:09:09.803 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:09:09.803 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:09:12.103 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:09:12.103 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:10:16.681 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:10:16.682 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:10:22.031 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:10:22.031 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:11:41.071 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:11:41.073 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:11:44.114 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:11:44.114 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:12:04.576 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:28:36.002 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.611 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:28:36.037 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.614 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:28:36.039 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.629 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:28:36.054 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.653 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:28:36.243 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.659 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:28:36.249 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.811 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:28:36.238 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.872 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:28:36.296 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:04.880 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:28:36.470 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:12:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:12:27.688 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:13:40.177 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:30:11.603 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.184 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:30:11.611 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.200 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:30:11.624 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.229 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:30:11.653 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.258 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:30:11.685 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.271 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:30:11.861 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.274 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:30:11.864 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.291 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:30:11.716 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:40.403 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:30:11.994 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:13:49.638 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:13:50.016 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:13:50.016 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:13:53.569 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:13:53.570 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:13:53.794 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:13:54.028 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:13:54.029 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:13:56.965 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:13:56.965 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:01.089 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:01.089 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:04.543 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:14:04.979 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:04.980 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:07.230 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:07.230 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:20.052 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:20.052 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:23.166 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:23.167 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:28.730 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:28.730 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:33.706 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:33.706 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:34.997 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:34.997 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:35.578 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:14:35.509+0000 43a69700 -1 handle_data failed with exit code -104 2019-12-08T16:14:36.366 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:14:36.687 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:36.687 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:39.336 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:40.216 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:40.216 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:14:42.164 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:42.291 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:45.289 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:45.289 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:47.156 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:47.156 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:49.957 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:49.957 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:57.353 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:57.354 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:57.871 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:57.871 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:14:58.899 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:14:59.412 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:14:59.412 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:02.569 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:02.570 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:05.781 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:05.782 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:18.605 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:18.605 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:23.692 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:23.692 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:27.981 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:27.981 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:28.973 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:28.973 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:29.568 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:15:29.559+0000 10f015700 -1 handle_data failed with exit code -104 2019-12-08T16:15:30.646 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:30.657 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:40.936 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:40.937 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:45.178 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:45.178 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:48.299 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:48.299 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:48.982 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:15:49.920 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:15:49.842+0000 12203b700 -1 handle_data failed with exit code -104 2019-12-08T16:15:50.798 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:15:50.798 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:00.891 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:00.892 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:04.291 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:10.062 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:10.063 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:11.272 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:11.273 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:11.742 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:16:11.722+0000 9d732700 -1 handle_data failed with exit code -104 2019-12-08T16:16:12.947 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:12.947 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:24.701 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:24.701 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:28.097 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:28.097 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:33.278 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:33.279 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:34.156 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:34.156 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:34.751 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:16:34.722+0000 69ecb700 -1 handle_data failed with exit code -104 2019-12-08T16:16:35.101 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:16:35.421 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:16:37.162 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:16:37.162 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:17:25.769 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:17:25.769 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:18:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:18:04.102 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:18:05.285 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:18:05.286 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:18:06.321 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:18:06.287+0000 108007700 -1 handle_data failed with exit code -104 2019-12-08T16:19:13.844 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:13.844 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:19:14.772 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:14.772 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:19:15.602 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:19:15.583+0000 d2f9d700 -1 handle_data failed with exit code -104 2019-12-08T16:19:48.109 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:19:48.668 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:48.668 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:19:51.594 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:51.594 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:19:55.022 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:55.022 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:19:56.041 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:56.041 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:19:57.481 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:19:57.482 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:20:23.068 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:20:23.068 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:20:50.616 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:20:50.616 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:21:14.947 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:21:14.947 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:21:15.982 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:21:15.983 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:21:34.982 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:21:34.982 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:21:36.210 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:21:36.210 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:21:36.758 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:21:36.729+0000 fafed700 -1 handle_data failed with exit code -104 2019-12-08T16:22:01.844 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:38:33.271 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:01.872 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:38:33.296 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:01.881 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:38:33.301 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:01.882 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:38:33.307 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:01.887 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:38:33.477 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:01.947 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:38:33.537 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:02.088 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:38:33.514 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:02.116 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:38:33.541 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:02.147 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:38:33.737 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:07.984 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:22:07.985 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:22:20.565 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:22:20.565 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:22:24.687 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:22:24.680+0000 f3fdf700 -1 handle_data failed with exit code -104 2019-12-08T16:22:25.104 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:22:25.104 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:22:25.897 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:22:25.897 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:22:26.800 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:22:26.765+0000 c4780700 -1 handle_data failed with exit code -104 2019-12-08T16:22:26.887 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:38:58.313 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.892 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:38:58.319 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.899 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:38:58.324 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.902 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:38:58.327 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.903 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:38:58.492 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.909 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:38:58.499 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.932 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:38:58.359 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.936 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:38:58.361 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:26.964 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:38:58.554 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:22:59.112 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:22:59.113 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:23:12.451 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:23:12.451 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:23:16.109 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:23:16.076+0000 5deb3700 -1 handle_data failed with exit code -104 2019-12-08T16:23:17.210 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:23:17.210 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:23:18.296 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:23:18.297 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:23:19.428 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:23:19.355+0000 5628e700 -1 handle_data failed with exit code -104 2019-12-08T16:23:52.643 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:23:52.700 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:23:52.682+0000 bc770700 -1 handle_data failed with exit code -104 2019-12-08T16:23:52.881 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:23:52.881 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:23:55.617 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:23:55.617 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:25:13.205 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:25:13.205 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:25:49.235 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:25:49.236 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:25:50.828 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:25:50.828 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:26:45.066 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:26:45.066 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:26:47.069 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:26:47.069 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:27:19.143 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:27:19.143 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:27:37.899 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:44:09.326 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:37.907 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:44:09.333 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:37.912 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:44:09.337 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:37.918 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:44:09.343 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:37.926 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:44:09.517 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:37.937 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:44:09.528 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:38.108 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:44:09.533 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:38.141 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:44:09.568 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:27:38.212 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:44:09.802 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:28:28.330 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:28:28.330 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:29:04.533 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:29:04.533 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:29:05.924 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:29:05.924 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:29:06.787 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:29:06.755+0000 104800700 -1 handle_data failed with exit code -104 2019-12-08T16:30:14.784 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:30:14.784 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:30:16.305 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:30:16.305 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:30:17.165 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:30:17.120+0000 ecfd1700 -1 handle_data failed with exit code -104 2019-12-08T16:30:23.644 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:46:55.070 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.685 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:46:55.112 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.694 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:46:55.119 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.703 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:46:55.127 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.734 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:46:55.325 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.739 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:46:55.330 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.778 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:46:55.205 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.814 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:46:55.238 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:23.839 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:46:55.430 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:30:50.760 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:30:50.760 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:31:57.506 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:31:57.507 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:32:35.268 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:32:35.268 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:32:36.803 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:32:36.776+0000 c7786700 -1 handle_data failed with exit code -104 2019-12-08T16:32:47.156 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:49:18.582 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.165 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:49:18.592 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.175 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:49:18.599 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.178 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:49:18.603 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.185 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:49:18.776 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.218 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:49:18.809 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.269 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:49:18.696 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.276 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:49:18.700 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:32:47.458 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:49:19.049 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:33:39.978 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:33:39.978 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:33:41.182 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:33:41.183 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:33:42.326 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:33:42.289+0000 13b06d700 -1 handle_data failed with exit code -104 2019-12-08T16:34:19.190 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:34:19.191 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:35:42.367 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:35:42.368 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:36:13.741 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:36:13.742 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:36:15.538 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:36:15.538 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:37:09.768 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:37:09.768 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:37:11.165 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:37:11.165 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:37:46.818 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:37:46.818 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:38:00.046 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:54:31.473 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.053 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:54:31.480 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.062 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:54:31.487 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.067 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:54:31.492 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.074 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:54:31.665 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.083 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:54:31.674 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.224 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:54:31.650 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.250 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:54:31.675 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:00.343 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:54:31.934 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.209 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:55:16.636 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.234 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:55:16.661 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.237 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:55:16.662 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.256 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:55:16.681 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.272 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:55:16.699 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.291 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:55:16.883 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.292 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:55:16.717 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.303 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:55:16.894 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:38:45.353 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:55:16.944 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:39:41.884 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:39:41.884 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:40:19.199 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:40:19.200 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:40:20.539 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:40:20.539 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:40:21.522 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:40:21.488+0000 49274700 -1 handle_data failed with exit code -104 2019-12-08T16:41:23.681 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:41:23.682 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:41:24.607 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:41:24.608 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:41:25.792 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:41:25.744+0000 fdff3700 -1 handle_data failed with exit code -104 2019-12-08T16:42:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:42:01.988 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:42:01.989 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:42:04.964 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:42:04.965 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:42:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:42:15.528 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:42:19.692 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:42:19.692 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:42:38.155 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:42:38.156 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:42:56.560 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:42:56.560 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:43:12.069 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:59:43.496 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.072 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:59:43.500 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.083 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:59:43.509 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.088 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:59:43.513 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.091 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:59:43.682 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.097 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:59:43.689 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.238 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:03:59:43.665 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.282 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:03:59:43.707 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:12.322 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:03:59:43.914 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:43:27.328 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:43:27.328 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:43:58.381 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:43:58.381 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:22.254 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:44:22.627 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:22.627 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:25.316 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:25.317 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:42.972 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:42.972 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:45.035 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:45.035 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:51.193 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:51.193 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:55.215 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:55.215 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:56.299 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:44:56.300 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:44:58.018 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:44:58.002+0000 ccf91700 -1 handle_data failed with exit code -104 2019-12-08T16:45:11.052 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:11.053 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:45:22.533 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:22.533 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:45:23.426 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:23.426 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:45:25.188 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:45:25.145+0000 4ea7f700 -1 handle_data failed with exit code -104 2019-12-08T16:45:34.987 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:34.987 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:45:44.818 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:44.818 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:45:45.765 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:45.765 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:45:46.225 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:45:46.200+0000 806f8700 -1 handle_data failed with exit code -104 2019-12-08T16:45:55.015 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:45:55.015 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:46:11.860 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:46:11.860 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:46:12.834 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:46:12.834 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:46:14.869 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:46:14.800+0000 13185a700 -1 handle_data failed with exit code -104 2019-12-08T16:46:33.851 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:46:33.851 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:46:46.813 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:46:46.814 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:46:48.036 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:46:48.037 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:46:48.761 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:46:48.737+0000 9271c700 -1 handle_data failed with exit code -104 2019-12-08T16:47:06.675 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:03:38.102 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.707 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:03:38.133 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.763 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:03:38.188 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.766 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:03:38.358 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.769 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:03:38.361 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.770 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:03:38.195 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.826 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:03:38.253 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.888 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:03:38.480 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:06.893 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:03:38.319 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:47:07.163 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:47:07.163 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:47:19.083 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:47:19.084 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:47:20.157 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:47:20.158 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:47:21.061 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T16:47:20.990+0000 125842700 -1 handle_data failed with exit code -104 2019-12-08T16:47:43.537 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:47:43.912 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:47:43.913 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:47:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:47:46.002 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:48:10.964 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:48:10.964 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:48:13.766 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:48:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:48:14.022 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:48:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:48:16.953 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:48:21.888 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:04:53.316 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.898 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:04:53.325 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.911 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:04:53.336 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.913 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:04:53.339 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.914 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:04:53.506 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.917 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:04:53.509 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.976 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:04:53.401 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:21.976 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:04:53.404 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:48:22.017 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:04:53.609 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:49:15.115 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:49:15.404 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:49:15.404 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:49:18.236 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:49:18.236 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:04.294 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:50:05.025 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:05.025 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:07.433 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:07.433 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:29.505 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:29.505 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:31.701 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:50:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:31.957 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:34.431 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:34.432 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:36.101 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:50:36.338 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:36.338 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:38.561 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:38.561 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:50:50.005 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:50:50.005 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:04.013 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:04.013 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:15.406 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:15.406 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:51:21.486 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:21.486 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:24.867 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:24.867 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:51:29.482 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:29.483 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:31.463 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:31.463 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:36.556 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:51:36.826 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:36.826 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:43.888 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:51:44.193 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:44.239 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:46.129 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:46.129 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:49.272 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:51:49.515 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:49.515 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:52.071 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:52.071 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:57.134 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:57.134 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:58.072 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:58.072 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:51:59.894 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:51:59.894 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:00.926 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:00.926 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:02.451 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:02.452 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:03.391 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:03.392 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:04.193 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:04.194 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:05.101 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:05.101 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:06.816 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:06.816 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:07.641 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:07.642 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:08.673 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:08.674 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:09.905 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:09.906 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:11.102 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:11.103 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:12.080 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:12.080 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:12.828 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:12.828 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:13.957 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:13.964 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:15.005 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:15.005 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:15.840 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:15.841 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:16.695 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:16.695 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:17.295 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:17.295 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:18.090 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:18.090 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:18.736 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:18.736 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:19.357 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:19.357 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:20.120 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:20.120 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:20.703 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:20.730 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:21.899 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:21.900 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:22.807 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:22.807 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:23.912 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:23.912 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:24.445 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:24.446 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:25.107 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:25.108 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:26.098 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:26.098 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:27.012 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:27.013 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:27.635 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:27.635 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:28.191 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:52:28.742 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:28.742 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:31.312 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:31.312 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:36.167 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:36.168 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:37.320 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:37.321 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:38.457 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:52:38.714 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:38.714 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:41.568 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:41.569 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:46.504 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:46.505 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:47.118 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:52:47.366 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:47.366 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:50.212 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:50.212 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:52.742 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:52.742 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:57.076 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:57.077 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:52:58.976 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:52:59.315 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:52:59.315 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:02.058 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:02.059 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:08.623 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:08.623 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:19.279 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:19.279 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:25.020 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:53:25.650 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:25.651 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:28.568 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:28.568 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:32.751 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:10:04.178 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.758 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:10:04.186 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.767 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:10:04.193 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.774 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:10:04.199 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.780 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:10:04.372 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.793 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:10:04.385 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.972 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:10:04.399 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:32.991 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:10:04.417 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:33.036 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:10:04.628 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:53:41.328 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:41.328 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:53:57.135 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:53:57.135 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:07.242 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:54:07.567 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:07.568 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:10.243 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:10.244 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:13.989 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:13.990 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:17.006 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:19.803 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:19.803 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:24.833 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:24.834 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:26.904 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:54:27.222 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:27.222 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:29.502 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:29.502 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:38.809 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:43.999 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:43.999 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:54:53.221 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:54:53.221 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:55:00.888 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:55:00.889 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:55:05.472 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:55:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:55:06.162 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:55:09.204 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:55:09.204 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:55:25.263 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:55:25.263 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:55:28.148 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:11:59.575 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.188 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:11:59.615 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.213 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:11:59.638 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.214 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:11:59.640 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.231 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:11:59.824 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.234 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:11:59.826 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.258 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:11:59.682 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.285 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:11:59.710 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:28.347 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:11:59.939 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:55:36.449 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:55:36.449 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:55:53.136 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:55:53.136 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:06.179 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:06.179 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:15.284 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:56:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:15.534 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:17.574 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:17.575 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:21.486 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:21.486 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:26.957 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:26.957 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:28.406 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:56:28.856 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:28.856 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:31.309 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:31.310 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:42.201 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:42.201 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:52.025 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:52.026 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:56.916 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:56:57.180 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:57.180 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:56:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:56:59.937 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:19.542 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:19.543 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:35.078 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:35.079 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:44.006 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:57:44.401 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:44.402 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:48.237 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:48.237 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:50.062 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:50.062 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:54.144 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:54.145 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:57:57.201 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:57:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:58:00.063 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:00.064 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:02.731 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:02.731 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:08.573 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:58:08.852 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:08.852 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:10.702 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:10.702 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:22.151 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:22.151 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:30.111 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:30.111 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:31.855 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:58:32.202 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:32.202 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:35.147 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:36.746 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:58:37.008 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:37.008 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:39.918 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:39.918 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:44.775 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:15:16.202 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.782 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:15:16.209 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.796 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:15:16.388 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.799 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:15:16.225 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.801 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:15:16.393 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.805 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:15:16.230 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.909 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:15:16.336 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.945 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:15:16.371 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:44.960 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:15:16.552 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T16:58:56.604 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:58:56.888 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:56.888 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:58:58.989 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:58:58.989 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:01.380 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:59:02.142 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:02.142 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:05.009 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:05.009 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:07.010 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:59:07.236 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:07.236 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:09.517 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:09.517 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:11.182 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:59:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:11.710 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:13.944 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:13.944 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:14.909 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:59:15.274 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:15.274 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:17.293 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:17.293 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:18.365 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:18.365 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:19.649 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:19.650 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:21.363 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:21.363 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:23.256 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:23.257 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:25.189 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T16:59:25.471 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:25.471 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T16:59:28.493 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T16:59:28.493 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:03:54.498 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:20:25.925 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.518 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:20:25.937 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.553 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:20:25.978 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.554 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:20:25.980 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.592 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:20:26.184 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.594 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:20:26.186 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.677 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:20:26.103 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.750 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:20:26.177 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:54.777 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:20:26.369 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.241 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:20:27.666 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.242 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:20:27.668 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.255 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:20:27.680 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.327 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:20:27.754 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.328 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:20:27.755 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.338 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:20:27.930 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.339 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:20:27.766 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.340 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:20:27.932 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:03:56.354 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:20:27.946 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:04:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:04:13.343 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:04:13.343 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:04:15.516 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:04:15.517 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:04.919 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:05:05.131 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:05.132 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:07.166 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:07.167 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:07.632 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:07.632 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:09.036 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:09.036 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:15.940 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:05:16.158 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:16.158 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:19.248 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:19.248 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:19.883 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:19.883 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:21.048 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:21.049 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:36.918 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:05:37.173 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:37.173 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:39.177 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:39.551 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:39.551 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:41.193 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:41.194 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:42.784 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:42.784 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:55.298 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:05:55.542 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:55.542 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:05:58.555 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:05:58.555 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:06:00.991 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:06:00.991 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:06:21.834 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:06:21.834 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:06:50.566 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:06:50.567 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:08:10.125 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:08:10.429 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:08:10.429 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:08:13.389 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:08:13.390 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:08:15.544 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:08:15.544 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:09:08.581 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:25:40.008 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.589 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:25:40.017 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.603 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:25:40.028 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.605 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:25:40.030 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.620 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:25:40.212 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.667 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:25:40.256 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.782 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:25:40.207 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.784 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:25:40.212 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:09:08.909 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:25:40.501 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:02.261 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:10:02.580 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:10:02.581 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:10:05.773 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:10:05.774 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:10:05.796 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:26:37.223 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.827 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:26:37.254 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.863 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:26:37.288 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.864 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:26:37.289 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.878 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:26:37.306 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.900 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:26:37.492 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.906 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:26:37.498 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.926 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:26:37.352 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:05.993 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:26:37.586 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:10:06.908 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:10:06.908 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:12:16.244 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:28:47.666 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.290 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:28:47.718 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.307 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:28:47.733 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.311 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:28:47.737 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.327 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:28:47.919 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.332 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:28:47.925 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.397 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:28:47.822 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.414 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:28:48.006 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:12:16.473 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:28:47.900 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.689 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:30:50.116 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.699 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:30:50.127 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.707 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:30:50.133 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.710 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:30:50.136 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.722 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:30:50.314 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.731 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:30:50.323 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.789 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:30:50.216 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.838 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:30:50.263 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:14:18.856 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:30:50.448 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:16:57.400 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:16:57.645 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:16:57.645 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:17:00.000 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:17:00.001 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:17:01.461 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:17:01.462 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:18:02.927 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:18:02.928 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:19:04.501 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:19:04.502 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:19:30.813 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:36:02.240 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:30.823 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:36:02.250 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:30.832 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:36:02.258 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:30.836 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:36:02.262 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:30.838 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:36:02.430 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:30.844 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:36:02.436 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:31.017 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:36:02.445 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:31.042 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:36:02.467 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:19:31.058 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:36:02.651 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:06.606 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:20:06.606 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:20:37.714 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:37:09.141 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.717 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:37:09.145 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.755 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:37:09.181 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.764 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:37:09.190 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.768 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:37:09.194 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.797 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:37:09.390 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.801 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:37:09.394 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.806 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:37:09.231 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:20:37.933 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:37:09.525 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:23:20.338 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:23:20.707 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:23:20.707 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:23:23.061 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:23:23.061 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:23:25.210 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:23:25.210 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:24:07.783 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:07.784 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:10.785 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:10.785 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:12.257 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:12.258 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:41.071 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:41:12.498 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.079 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:41:12.506 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.085 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:41:12.510 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.087 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:41:12.513 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.106 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:41:12.697 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.112 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:41:12.705 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:41.203 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:41.275 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:41:12.689 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.288 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:41:12.716 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:41.296 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:41:12.888 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:24:53.431 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:24:53.694 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:53.694 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:56.221 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:56.222 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:24:58.189 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:24:58.189 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:14.309 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:25:14.662 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:14.662 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:17.396 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:17.397 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:18.889 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:40.958 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:25:41.229 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:41.229 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:43.826 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:43.826 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:45.489 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:45.489 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:47.586 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:25:47.937 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:47.938 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:50.444 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:50.444 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:25:52.185 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:25:52.185 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:07.687 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:26:07.914 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:07.915 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:09.908 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:11.500 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:11.500 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:23.812 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:23.812 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:26:26.324 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:26.324 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:28.210 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:28.210 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:29.604 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:29.605 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:29.886 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:29.886 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:30.051 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:30.051 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:30.441 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:30.442 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:30.670 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:26:30.670 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:26:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-53: 2019-12-08T17:26:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:32.820 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:32.820 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:32.820 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:32.820 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:32.820 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:32.821 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:33.111 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-51: 2019-12-08T17:26:33.111 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:33.111 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:33.111 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:33.112 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:33.113 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:33.113 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:33.113 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:33.123 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-55: 2019-12-08T17:26:33.123 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:33.123 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:33.123 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:33.123 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:33.123 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:33.124 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:33.125 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:33.321 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-54: 2019-12-08T17:26:33.321 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:33.321 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:33.321 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:33.321 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:33.322 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:33.323 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:33.323 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-62: 2019-12-08T17:26:43.518 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:43.518 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:43.518 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:43.519 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:43.520 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:43.520 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:43.520 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:43.685 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-65: 2019-12-08T17:26:43.685 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:43.685 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:43.685 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:43.685 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:43.685 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:43.686 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:43.687 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:54.507 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-72: 2019-12-08T17:26:54.507 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:54.507 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:54.507 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:54.507 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:54.507 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:54.508 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:54.508 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:54.508 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:54.508 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:54.508 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:54.508 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:54.509 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:54.509 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:54.783 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-74: 2019-12-08T17:26:54.783 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:54.783 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:54.784 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:54.785 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:54.785 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:54.785 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:54.785 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:54.789 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-75: 2019-12-08T17:26:54.789 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:54.789 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:54.789 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:54.789 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:54.790 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:54.790 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:54.790 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:54.790 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:54.790 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:54.790 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:54.791 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:54.791 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:54.791 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:26:54.822 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-71: 2019-12-08T17:26:54.822 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:26:54.822 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:26:54.822 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:26:54.822 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:26:54.822 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:26:54.823 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:26:54.824 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:04.794 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-83: 2019-12-08T17:27:04.794 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:04.794 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:04.794 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:04.795 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:04.795 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:04.795 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:04.795 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:04.795 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:04.795 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:04.796 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:04.796 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:04.796 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-84: 2019-12-08T17:27:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:05.313 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:05.313 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:05.313 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:05.314 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:05.315 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:05.315 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:05.387 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-82: 2019-12-08T17:27:05.387 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:05.387 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:05.387 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:05.387 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:05.388 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:05.388 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:05.388 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:05.388 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:05.388 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:05.389 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:05.389 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:05.389 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:05.532 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-81: 2019-12-08T17:27:05.532 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:05.532 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:05.532 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:05.532 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:05.532 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:05.533 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:05.534 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:14.999 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-92: 2019-12-08T17:27:14.999 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:14.999 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:14.999 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:15.000 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:15.000 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:15.000 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:15.000 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:15.000 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:15.000 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:15.001 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:15.001 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:15.001 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:15.001 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:15.179 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-95: 2019-12-08T17:27:15.180 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:15.180 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:15.180 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:15.180 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:15.180 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:15.180 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:15.181 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:15.664 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-91: 2019-12-08T17:27:15.664 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:15.665 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:15.665 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:15.665 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:15.665 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:15.665 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:15.666 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:15.666 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:15.666 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:15.666 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:15.666 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:15.666 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:15.667 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:15.700 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-93: 2019-12-08T17:27:15.700 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:15.700 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:15.701 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:15.701 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:15.701 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:15.701 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:15.701 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:15.701 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:15.702 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:15.702 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:15.702 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:15.702 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:15.702 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:24.964 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:27:25.505 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:25.506 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:27.837 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:27.837 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:29.665 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:29.666 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:29.897 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:29.897 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:30.207 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:30.207 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:30.687 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:30.688 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:31.464 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:31.465 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:31.776 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:31.777 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:32.053 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:32.053 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:32.833 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:32.834 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:33.564 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:33.564 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:35.394 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-101: 2019-12-08T17:27:35.394 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:35.394 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:35.394 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:35.394 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:35.394 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:35.395 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:35.396 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:35.420 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-103: 2019-12-08T17:27:35.420 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:35.420 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:35.420 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:35.421 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:35.422 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:35.422 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:35.422 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:35.422 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:35.533 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-104: 2019-12-08T17:27:35.533 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:35.533 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:35.533 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:35.534 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:35.535 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:35.535 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:35.535 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:35.840 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-106: 2019-12-08T17:27:35.841 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:35.841 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:35.841 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:35.841 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:35.842 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:35.842 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:35.842 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:35.842 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:35.843 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:35.843 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:35.843 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:35.843 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:35.844 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:35.883 INFO:teuthology.orchestra.run.smithi129.stderr:Exception in thread Thread-105: 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr:Traceback (most recent call last): 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr: self.run() 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-08T17:27:35.884 INFO:teuthology.orchestra.run.smithi129.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-08T17:27:35.885 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-08T17:27:35.885 INFO:teuthology.orchestra.run.smithi129.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-08T17:27:35.885 INFO:teuthology.orchestra.run.smithi129.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-08T17:27:35.885 INFO:teuthology.orchestra.run.smithi129.stderr: raise error_class(parsed_response, operation_name) 2019-12-08T17:27:35.885 INFO:teuthology.orchestra.run.smithi129.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-08T17:27:35.885 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T17:27:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:27:47.044 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:47.044 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:49.817 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:49.817 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:52.043 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:27:52.699 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:52.699 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:55.140 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:27:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:27:57.036 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:27:57.036 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:00.090 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:00.090 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:02.308 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:02.580 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:02.581 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:05.031 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:05.244 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:05.836 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:05.836 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:08.727 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:08.727 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:09.043 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:09.300 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:09.300 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:12.623 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:12.623 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:13.702 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:14.067 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:14.067 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:16.458 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:16.458 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:23.419 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:23.830 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:23.830 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:26.263 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:26.412 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:26.689 INFO:teuthology.orchestra.run.smithi129.stderr:Check for valid x-amz-expiration header after PUT ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:26.689 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:29.143 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:29.143 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:33.268 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:33.545 INFO:teuthology.orchestra.run.smithi129.stderr:Check for valid x-amz-expiration header on HEAD request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:33.545 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:36.551 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:36.551 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:43.074 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:43.074 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:43.590 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:44.014 INFO:teuthology.orchestra.run.smithi129.stderr:Check for valid x-amz-expiration header on GET request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:44.014 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:45.583 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:45.583 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:49.511 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:49.512 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:49.924 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:28:50.194 INFO:teuthology.orchestra.run.smithi129.stderr:Check x-amz-expiration-header object tags interaction ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:50.194 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:28:52.354 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:28:59.188 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:45:30.615 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.192 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:45:30.620 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.193 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:45:30.619 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.214 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:45:30.640 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.218 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:45:30.810 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.230 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:45:30.822 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.332 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:45:30.756 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.342 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:45:30.768 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:28:59.396 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:45:30.988 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:00.258 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:00.562 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:05.341 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:05.342 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:07.548 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:07.805 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:07.805 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:10.840 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:10.840 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:12.090 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:12.361 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:12.361 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:15.416 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:15.416 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:17.470 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:17.744 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:17.745 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:21.363 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:21.364 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:23.329 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:26.909 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:26.909 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:29.071 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:29.373 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:29.373 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:31.977 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:31.977 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:33.347 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:33.932 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:33.932 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:36.484 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:36.485 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:37.742 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:38.022 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:38.022 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:40.742 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:40.742 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:45.589 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:46.092 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:46.092 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:49.028 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:49.028 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:50.893 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:51.146 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:51.147 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:53.247 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:46:24.674 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.257 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:46:24.685 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.267 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:46:24.693 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.270 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:46:24.695 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.275 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:46:24.867 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.281 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:46:24.873 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.330 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:46:24.755 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.345 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:46:24.772 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.369 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:46:24.961 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:29:53.795 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:53.796 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:55.614 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:29:55.850 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:55.850 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:29:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:29:58.689 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:00.619 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:30:00.986 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:00.986 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:03.515 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:03.515 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:05.729 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:30:06.585 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:06.585 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:09.327 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:09.328 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:09.496 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:30:09.736 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:09.736 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:12.142 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:12.142 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:12.432 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:30:12.681 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:12.681 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:15.185 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:15.186 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:15.337 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:30:15.540 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:15.540 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:18.374 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:18.374 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:18.571 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:30:18.829 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:18.830 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:30:21.361 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:30:21.361 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:32:01.759 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:32:01.759 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:33:11.067 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:33:11.067 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:33:51.942 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:33:52.321 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:33:52.321 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:33:54.559 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:33:54.559 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:05.259 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:51:36.686 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.265 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:51:36.693 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.273 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:51:36.698 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.277 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:51:36.703 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.297 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:51:36.889 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.306 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:51:36.899 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.458 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:51:36.885 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.512 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:51:36.937 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:05.553 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:51:37.144 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:35:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:06.739 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:07.070 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:35:07.384 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:07.384 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:10.167 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:10.167 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:12.398 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:12.398 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:12.740 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:35:13.016 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:13.016 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:15.160 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:15.161 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:35:18.856 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:18.857 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:21.392 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:21.393 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:23.332 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:35:23.791 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:23.791 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:26.333 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:26.333 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:26.473 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:35:26.905 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:26.906 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:29.614 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:29.615 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:30.160 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:35:30.474 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:30.474 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:35:32.433 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:35:32.433 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:37:04.915 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:37:04.916 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:37:07.114 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:37:07.115 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:37:20.533 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:53:51.961 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.560 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:53:51.988 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.564 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:53:51.990 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.589 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:53:52.014 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.621 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:53:52.213 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.630 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:53:52.223 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.650 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:53:52.077 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.665 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:53:52.091 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:37:20.729 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:53:52.312 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:38:07.106 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:38:07.106 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:38:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:38:09.505 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:38:49.149 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:38:49.389 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:38:49.389 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:38:51.950 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:38:51.950 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:39:56.975 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:39:57.239 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:39:57.239 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:39:59.658 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:39:59.658 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:40:16.001 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:56:47.428 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.003 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:56:47.431 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.028 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:56:47.453 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.033 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:56:47.459 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.042 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:56:47.635 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.058 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:56:47.651 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.120 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:04:56:47.548 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.187 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:04:56:47.613 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:40:16.244 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:04:56:47.836 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:41:02.584 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:03.159 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:06.086 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:06.086 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:07.687 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:07.687 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:08.615 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:08.897 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:11.328 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:11.328 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:13.472 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:13.774 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:13.774 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:16.771 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:16.771 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:19.055 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:19.407 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:19.408 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:22.576 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:22.576 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:26.612 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:27.369 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:27.369 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:30.045 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:30.045 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:32.921 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:33.318 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:33.318 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:36.653 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:36.653 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:38.479 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:38.876 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:40.654 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:40.655 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:44.148 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:45.824 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:46.113 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:46.113 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:48.457 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:48.457 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:52.229 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:52.229 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:53.431 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:41:53.725 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:53.725 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:55.625 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:55.625 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:41:59.672 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:41:59.672 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:03.503 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:42:03.736 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:03.736 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:05.686 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:05.686 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:09.246 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:09.246 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:12.261 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:42:12.516 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:12.516 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:14.497 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:14.497 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:17.889 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:17.890 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:23.631 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:23.631 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:24.786 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:24.786 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:25.155 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:42:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:25.693 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:28.285 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:28.285 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:30.947 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:30.947 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:36.595 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:36.595 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:37.434 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:37.434 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:38.014 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:42:38.554 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:38.554 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:40.883 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:40.883 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:53.092 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:53.092 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:54.810 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:42:55.099 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:55.099 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:42:57.140 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:42:57.140 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:10.859 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:10.859 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:12.538 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:43:12.838 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:12.838 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:15.143 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:15.143 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:28.324 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:28.324 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:29.653 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:43:30.194 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:30.194 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:31.991 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:31.992 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:42.769 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:42.770 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:43.839 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:43:44.118 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:44.118 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:46.213 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:58.348 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:43:58.348 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:43:59.990 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:44:00.305 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:00.305 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:03.635 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:03.635 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:17.738 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:17.738 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:18.302 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:44:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:18.553 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:20.233 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:20.233 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:34.992 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:34.992 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:35.531 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:44:35.831 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:35.831 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:38.964 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:38.964 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:51.579 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:51.579 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:54.496 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:44:54.926 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:54.927 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:44:57.469 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:44:57.542 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:12.227 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:12.227 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:15.421 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:45:15.752 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:15.753 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:18.988 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:18.988 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:21.724 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:45:22.121 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:22.122 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:24.544 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:24.545 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:26.984 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:01:58.411 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:26.989 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:01:58.416 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:26.997 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:01:58.423 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.004 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:01:58.427 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.011 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:01:58.604 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.028 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:01:58.620 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:27.073 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:27.162 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:01:58.590 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.187 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:01:58.613 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.265 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:01:58.857 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:27.370 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:45:27.601 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:27.601 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:30.121 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:30.121 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:31.831 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:45:32.077 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:32.077 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:34.955 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:34.955 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:42.032 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:02:13.459 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.036 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:02:13.464 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.091 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:02:13.517 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.094 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:02:13.520 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.105 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:02:13.533 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.120 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:02:13.546 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.123 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:02:13.715 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.128 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:02:13.720 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:42.163 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:02:13.753 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:45:43.316 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:43.316 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:46.766 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:46.767 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:47.423 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:45:47.791 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:47.791 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:49.906 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:49.907 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:45:58.723 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:45:58.724 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:00.121 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:46:01.284 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:01.284 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:03.854 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:03.854 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:15.128 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:15.128 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:18.325 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:18.325 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:19.735 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:46:20.174 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:20.174 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:22.165 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:22.165 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:23.379 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:23.380 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:26.255 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:26.255 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:28.946 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:28.946 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:31.909 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:31.909 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:34.603 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:46:34.812 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:34.812 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:36.847 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:36.847 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:46:37.882 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:46:37.882 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:47:35.913 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:47:35.916 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:48:40.531 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:48:40.574 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:48:43.768 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:48:43.769 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:49:43.627 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:49:43.871 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:49:43.871 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:49:50.238 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:49:50.238 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:49:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:49:55.077 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:49:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:49:55.780 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:49:56.724 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:49:57.024 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:49:57.024 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:04.369 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:04.369 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:09.047 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:09.047 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:11.091 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:50:11.443 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:11.443 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:19.409 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:19.409 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:23.859 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:23.860 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:27.306 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:50:27.606 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:27.607 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:36.070 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:36.070 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:37.782 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:37.783 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:37.981 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:07:09.409 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:37.989 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:07:09.416 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:37.997 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:07:09.422 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:37.999 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:07:09.425 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:38.013 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:07:09.605 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:38.033 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:07:09.625 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:38.186 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:07:09.612 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:38.202 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:07:09.630 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:38.351 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:07:09.943 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:41.088 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:41.088 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:44.776 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:50:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:45.138 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:50.818 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:50.818 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:52.147 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:52.147 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:56.597 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:56.597 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:50:58.108 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:07:29.536 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.112 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:07:29.540 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.184 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:07:29.611 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.197 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:07:29.789 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.201 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:07:29.793 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.227 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:07:29.653 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.235 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:07:29.828 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.236 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:07:29.661 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:58.270 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:07:29.696 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:50:59.134 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:50:59.371 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:50:59.372 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:03.028 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:03.028 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:04.109 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:04.109 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:06.126 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:06.472 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:06.473 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:08.832 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:08.832 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:10.708 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:10.708 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:13.939 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:13.940 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:16.819 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:17.176 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:17.176 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:28.903 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:28.903 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:30.155 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:30.608 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:30.608 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:34.435 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:34.994 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:34.994 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:39.021 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:39.021 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:41.880 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:42.310 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:44.476 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:45.026 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:45.026 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:48.400 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:48.753 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:48.753 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:51.713 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:51.985 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:51.985 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:54.598 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:51:54.815 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:51:54.815 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:51:59.817 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:00.027 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:00.028 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:03.796 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:04.166 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:04.166 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:17.612 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:17.887 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:17.887 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:20.884 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:21.309 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:21.310 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:28.225 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:28.548 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:28.549 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:39.981 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:40.286 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:40.287 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:45.501 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:45.744 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:45.744 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:52:58.988 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:52:59.346 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:52:59.347 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:53:11.987 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:53:12.222 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:53:12.223 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:53:23.830 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:53:24.194 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:53:24.194 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:53:37.597 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:53:37.855 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:53:37.856 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:53:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:53:54.011 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:53:54.011 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:03.673 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:10:35.101 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.721 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:10:35.148 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.730 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:10:35.156 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.731 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:10:35.157 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.757 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:10:35.349 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.761 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:10:35.353 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.817 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:10:35.243 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.871 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:10:35.463 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:03.879 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:10:35.306 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:54:07.626 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:07.857 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:07.858 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:14.883 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:15.120 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:15.121 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:21.383 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:21.619 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:21.619 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:27.101 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:27.472 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:27.472 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:40.279 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:40.558 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:43.995 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:44.231 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:44.232 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:54:54.920 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:54:55.147 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:54:55.147 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:55:06.039 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:55:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:55:06.284 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:55:19.682 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:55:20.063 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:55:20.063 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:55:34.591 INFO:teuthology.orchestra.run.smithi129.stderr:ok 2019-12-08T17:55:34.607 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-12-08T17:55:34.837 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T17:55:34.837 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T17:55:49.123 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:12:20.551 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.130 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:12:20.558 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.140 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:12:20.566 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.141 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:12:20.733 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.144 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:12:20.737 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.145 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:12:20.571 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.206 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:12:20.632 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.221 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:12:20.645 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T17:55:49.238 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:12:20.830 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.012 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:17:32.439 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.038 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:17:32.444 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.038 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:17:32.455 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.038 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:17:32.459 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.042 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:17:32.633 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.047 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:17:32.640 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.213 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:17:32.641 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.220 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:17:32.646 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:01:01.236 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:17:32.829 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.285 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:18:56.713 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.325 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:18:56.753 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.344 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:18:56.937 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.347 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:18:56.940 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.352 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:18:56.777 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.406 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:18:56.832 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.414 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:18:56.842 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.415 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:18:57.008 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:02:25.478 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:18:56.904 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.421 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:22:45.846 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.428 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:22:45.856 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.441 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:22:45.867 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.448 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:22:46.040 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.449 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:22:45.875 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.466 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:22:46.058 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.588 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:22:46.014 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.628 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:22:46.055 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:06:14.669 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:22:46.261 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.753 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:27:18.180 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.782 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:27:18.210 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.813 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:27:18.237 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.820 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:27:18.246 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.823 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:27:18.416 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.825 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:27:18.418 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.913 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:27:18.341 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:46.987 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:27:18.579 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:10:47.025 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:27:18.450 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.701 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:27:57.128 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.709 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:27:57.137 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.721 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:27:57.147 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.726 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:27:57.152 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.726 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:27:57.319 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.736 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:27:57.329 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.771 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:27:57.198 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.771 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:27:57.196 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:11:25.816 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:27:57.408 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:37.953 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:33:09.372 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:37.958 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:33:09.385 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:37.967 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:33:09.393 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:37.972 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:33:09.398 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:37.976 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:33:09.568 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:37.992 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:33:09.585 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:38.133 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:33:09.561 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:38.204 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:33:09.629 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:16:38.268 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:33:09.860 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:48.980 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:38:20.407 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.001 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:38:20.415 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.001 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:38:20.422 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.001 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:38:20.426 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.002 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:38:20.594 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.017 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:38:20.610 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.165 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:38:20.590 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.201 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:38:20.793 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:21:49.204 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:38:20.632 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:02.999 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:43:34.427 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.020 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:43:34.433 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.021 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:43:34.442 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.021 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:43:34.446 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.024 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:43:34.617 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.035 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:43:34.628 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.241 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:43:34.668 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.244 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:43:34.670 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:03.262 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:43:34.854 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.875 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:44:06.303 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.896 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:44:06.324 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.941 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:44:06.367 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.942 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:44:06.368 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.955 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:44:06.547 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.957 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:44:06.549 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.974 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:44:06.402 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:34.986 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:44:06.412 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:27:35.037 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:44:06.630 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:15.898 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:48:47.326 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:15.941 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:48:47.369 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.004 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:48:47.430 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.007 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:48:47.432 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.009 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:48:47.602 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.012 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:48:47.605 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.106 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:48:47.534 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.198 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:48:47.624 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:16.257 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:48:47.849 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.594 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:48:49.020 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.595 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:48:49.021 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.601 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:48:49.193 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.602 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:48:49.195 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.603 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:48:49.031 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.605 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:48:49.032 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.620 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:48:49.210 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.621 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:48:49.038 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:32:17.631 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:48:49.055 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.719 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:52:28.146 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.763 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:52:28.190 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.792 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:52:28.214 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.792 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:52:28.217 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.806 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:52:28.399 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.813 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:52:28.406 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.863 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:52:28.291 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.913 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:52:28.338 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:35:56.961 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:52:28.554 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.111 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:53:58.538 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.117 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:53:58.545 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.126 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:53:58.552 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.132 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:53:58.724 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.133 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:53:58.559 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.141 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:53:58.734 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.199 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:53:58.627 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.213 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:53:58.639 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:37:27.238 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:53:58.831 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.536 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:59:10.964 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.541 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:59:10.969 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.550 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:59:10.976 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.556 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:59:10.982 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.568 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:59:11.160 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.575 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:59:11.168 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.730 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:05:59:11.158 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.743 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:05:59:11.169 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:42:39.869 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:05:59:11.462 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.343 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:00:49.770 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.371 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:00:49.772 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.384 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:00:49.809 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.403 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:00:49.830 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.411 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:00:49.837 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.433 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:00:50.025 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.435 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:00:50.028 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.464 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:00:49.890 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:44:18.570 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:00:50.163 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.043 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:04:20.471 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.052 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:04:20.480 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.062 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:04:20.488 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.064 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:04:20.656 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.068 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:04:20.661 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.069 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:04:20.495 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.200 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:04:20.792 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.205 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:04:20.631 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:47:49.240 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:04:20.667 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:39.894 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:09:11.321 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:39.934 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:09:11.362 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:39.987 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:09:11.413 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:39.998 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:09:11.424 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:40.016 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:09:11.608 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:40.019 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:09:11.612 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:40.059 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:09:11.487 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:40.152 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:09:11.578 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:52:40.164 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:09:11.757 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.038 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:09:31.466 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.043 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:09:31.470 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.071 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:09:31.664 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.078 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:09:31.504 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.081 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:09:31.509 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.082 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:09:31.508 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.085 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:09:31.678 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.115 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:09:31.541 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:53:00.161 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:09:31.753 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.011 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:14:45.439 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.037 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:14:45.625 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.038 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:14:45.629 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.038 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:14:45.442 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.068 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:14:45.494 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.070 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:14:45.496 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.199 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:14:45.627 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.226 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:14:45.817 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T18:58:14.248 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:14:45.673 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.469 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:17:32.896 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.498 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:17:32.926 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.531 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:17:32.957 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.537 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:17:33.130 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.538 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:17:32.964 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.549 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:17:33.141 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.580 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:17:33.008 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.638 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:17:33.064 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:01:01.640 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:17:33.233 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.005 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:19:54.432 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.013 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:19:54.441 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.022 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:19:54.447 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.025 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:19:54.451 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.031 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:19:54.624 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.041 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:19:54.634 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.111 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:19:54.538 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.118 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:19:54.544 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:03:23.190 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:19:54.783 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.402 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:25:06.829 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.411 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:25:06.839 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.419 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:25:06.845 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.423 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:25:06.849 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.432 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:25:07.025 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.445 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:25:07.038 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.593 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:25:07.020 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.618 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:25:07.040 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:08:35.642 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:25:07.235 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:22.925 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:25:54.352 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:22.960 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:25:54.387 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:22.972 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:25:54.398 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:22.975 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:25:54.401 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:22.985 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:25:54.574 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:22.985 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:25:54.576 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:23.008 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:25:54.434 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:23.027 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:25:54.453 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:09:23.040 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:25:54.633 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.787 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:29:40.215 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.792 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:29:40.220 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.833 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:29:40.258 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.853 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:29:40.279 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.858 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:29:40.451 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.885 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:29:40.478 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.977 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:29:40.403 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:08.984 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:29:40.411 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:09.084 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:29:40.676 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.034 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:30:16.462 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.039 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:30:16.466 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.053 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:30:16.478 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.059 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:30:16.485 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.066 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:30:16.658 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.071 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:30:16.664 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.095 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:30:16.520 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.111 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:30:16.537 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:13:45.180 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:30:16.773 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.565 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:34:15.992 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.601 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:34:16.025 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.642 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:34:16.067 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.642 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:34:16.068 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.672 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:34:16.265 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.687 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:34:16.280 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.713 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:34:16.141 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.751 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:34:16.177 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:17:44.877 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:34:16.470 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:56.973 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:35:28.400 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:56.979 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:35:28.406 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:56.991 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:35:28.584 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:56.992 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:35:28.417 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:56.996 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:35:28.589 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:56.997 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:35:28.423 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:57.050 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:35:28.478 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:57.062 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:35:28.488 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:18:57.093 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:35:28.686 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.723 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:40:42.150 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.746 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:40:42.155 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.746 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:40:42.164 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.748 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:40:42.341 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.750 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:40:42.176 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.753 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:40:42.346 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:10.981 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:40:42.409 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:11.003 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:40:42.595 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:24:11.007 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:40:42.430 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.037 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:42:37.464 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.044 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:42:37.472 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.063 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:42:37.655 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.101 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:42:37.515 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.101 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:42:37.516 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.107 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:42:37.534 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.125 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:42:37.717 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.156 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:42:37.582 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:26:06.190 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:42:37.783 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.638 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:45:53.065 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.656 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:45:53.083 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.662 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:45:53.255 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.662 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:45:53.088 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.664 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:45:53.090 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.667 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:45:53.260 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.775 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:45:53.202 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.800 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:45:53.220 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:29:21.814 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:45:53.406 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.513 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:50:58.940 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.522 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:50:58.947 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.527 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:50:58.955 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.543 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:50:58.969 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.572 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:50:59.165 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.573 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:50:59.166 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.679 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:50:59.107 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.690 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:50:59.115 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:27.698 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:50:59.291 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.132 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:51:02.560 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.137 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:51:02.564 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.145 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:51:02.571 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.149 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:51:02.575 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.151 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:51:02.744 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.157 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:51:02.750 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.162 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:51:02.589 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.176 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:51:02.602 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:34:31.188 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:51:02.780 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:36:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T19:36:04.344 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T19:36:13.293 INFO:teuthology.orchestra.run.smithi129.stderr:/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/urllib3/connection.py:418: SubjectAltNameWarning: Certificate for smithi129.front.sepia.ceph.com has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) 2019-12-08T19:36:13.294 INFO:teuthology.orchestra.run.smithi129.stderr: SubjectAltNameWarning, 2019-12-08T19:36:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-08T19:36:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-08T19:36:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-08T19:36:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-08T19:36:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-08T19:36:14.944 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-08T19:36:14.944 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-08T19:36:15.338 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-08T19:36:15.512 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-08T19:36:15.844 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-08T19:36:15.873 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-08T19:36:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-08T19:36:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-08T19:36:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-08T19:36:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-08T19:36:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-08T19:36:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-08T19:36:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-08T19:36:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-08T19:36:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-08T19:36:15.894 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-08T19:36:15.894 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-08T19:36:15.898 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-08T19:36:15.898 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-08T19:36:15.898 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-08T19:36:15.899 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-08T19:36:15.903 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-08T19:36:15.903 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-08T19:36:15.903 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-08T19:36:15.903 INFO:teuthology.orchestra.run.smithi129.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-08T19:36:15.924 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-08T19:36:15.945 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-08T19:36:15.945 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-08T19:36:16.098 INFO:teuthology.orchestra.run.smithi129.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-08T19:36:16.099 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T19:36:16.099 INFO:teuthology.orchestra.run.smithi129.stderr:---------------------------------------------------------------------- 2019-12-08T19:36:16.099 INFO:teuthology.orchestra.run.smithi129.stderr:Ran 611 tests in 16169.267s 2019-12-08T19:36:16.099 INFO:teuthology.orchestra.run.smithi129.stderr: 2019-12-08T19:36:16.099 INFO:teuthology.orchestra.run.smithi129.stderr:OK (SKIP=75) 2019-12-08T19:36:16.145 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-12-08T19:36:16.146 INFO:teuthology.orchestra.run.smithi129:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-12-08T19:36:16.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-08T19:36:16.264 INFO:tasks.s3tests:Cleaning up boto... 2019-12-08T19:36:16.265 INFO:teuthology.orchestra.run.smithi129:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-08T19:36:16.306 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-12-08T19:36:16.478 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.476+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.503+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.503+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.522 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.519+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.522 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.519+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.528 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.525+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.528 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.525+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.535 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.532+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:16.715 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.712+0000 7fd3c5c63ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T19:36:16.715 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.712+0000 7fd3c5c63ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T19:36:16.715 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.712+0000 7fd3c5c63ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T19:36:16.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.712+0000 7fd3c5c63ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T19:36:16.716 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.712+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.726 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.719+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:16.726 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.719+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.727+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:16.729 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.727+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.734+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.734+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.758 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.756+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:16.758 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.756+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.763 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.761+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.763 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.761+0000 7fd3c5c63ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T19:36:16.763 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.761+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.767+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.770 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.767+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.778+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:16.780 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.778+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.794+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:16.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.794+0000 7fd3c5c63ec0 20 zonegroup default 2019-12-08T19:36:16.797 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.794+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.804 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.802+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.804 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.802+0000 7fd3c5c63ec0 10 Cannot find current period zone using local zone 2019-12-08T19:36:16.804 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.802+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.807+0000 7fd3c5c63ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:16.809 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.807+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.815+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:16.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.815+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:16.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.827+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:16.829 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.827+0000 7fd3c5c63ec0 20 zone default found 2019-12-08T19:36:16.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.827+0000 7fd3c5c63ec0 20 started sync module instance, tier type = 2019-12-08T19:36:16.830 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:16.827+0000 7fd3c5c63ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T19:36:17.262 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.259+0000 7fd3c5c63ec0 20 add_watcher() i=0 2019-12-08T19:36:17.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.259+0000 7fd3c5c63ec0 20 add_watcher() i=1 2019-12-08T19:36:17.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.259+0000 7fd3c5c63ec0 20 add_watcher() i=2 2019-12-08T19:36:17.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.259+0000 7fd3c5c63ec0 20 add_watcher() i=3 2019-12-08T19:36:17.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.259+0000 7fd3c5c63ec0 20 add_watcher() i=4 2019-12-08T19:36:17.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.259+0000 7fd3c5c63ec0 20 add_watcher() i=5 2019-12-08T19:36:17.267 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.265+0000 7fd3c5c63ec0 20 add_watcher() i=6 2019-12-08T19:36:17.299 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.297+0000 7fd3c5c63ec0 20 add_watcher() i=7 2019-12-08T19:36:17.300 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.297+0000 7fd3c5c63ec0 2 all 8 watchers are set, enabling cache 2019-12-08T19:36:17.300 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.297+0000 7fd3837de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T19:36:17.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.299+0000 7fd3b368e700 0 -- >> v1:172.21.15.181:6789/0 conn(0x55a154e882c0 legacy=0x55a154e8b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-08T19:36:17.359 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:17.356+0000 7fd3b2e8d700 0 -- 172.21.15.129:0/2081376317 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7fd2f4022480 msgr2=0x7fd2f4024990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:18.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.893+0000 7fd2f1ffb700 20 reqs_thread_entry: start 2019-12-08T19:36:18.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.893+0000 7fd2eb7fe700 20 reqs_thread_entry: start 2019-12-08T19:36:18.896 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.893+0000 7fd3c5c63ec0 20 init_complete bucket index max shards: 0 2019-12-08T19:36:18.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.894+0000 7fd2e9ffb700 20 reqs_thread_entry: start 2019-12-08T19:36:18.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.894+0000 7fd3c5c63ec0 20 get_system_obj_state: rctx=0x7fff3a635d88 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55a154f0d0d0 s->prefetch_data=0 2019-12-08T19:36:18.897 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.894+0000 7fd3c5c63ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-08T19:36:18.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.913+0000 7fd3c5c63ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-08T19:36:18.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.913+0000 7fd3c5c63ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:18.915 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.913+0000 7fd3c5c63ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T19:36:18.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.913+0000 7fd3c5c63ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T19:36:18.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.913+0000 7fd3c5c63ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T19:36:18.916 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.913+0000 7fd3c5c63ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:18.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.938+0000 7fd3c5c63ec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-08T19:36:18.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.938+0000 7fd3c5c63ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-08T19:36:18.944 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.938+0000 7fd3c5c63ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:18.959 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.957+0000 7fd3c5c63ec0 10 removing key index: OWGENUEYNKRCMKZNLSID 2019-12-08T19:36:18.959 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.957+0000 7fd3c5c63ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=170 2019-12-08T19:36:18.976 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:18.974+0000 7fd393fff700 10 RGWWatcher::handle_notify() notify_id 575525617720 cookie 94151402533584 notifier 4706 bl.length()=170 2019-12-08T19:36:19.195 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.192+0000 7fd3c5c63ec0 10 removing email index: bar.client.0+test@test.test 2019-12-08T19:36:19.195 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.192+0000 7fd3c5c63ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-12-08T19:36:19.206 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.203+0000 7fd393fff700 10 RGWWatcher::handle_notify() notify_id 575525617724 cookie 94151402527744 notifier 4706 bl.length()=178 2019-12-08T19:36:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.499+0000 7fd3c5c63ec0 10 removing user buckets index 2019-12-08T19:36:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.499+0000 7fd3c5c63ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-12-08T19:36:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.518+0000 7fd393fff700 10 RGWWatcher::handle_notify() notify_id 575525618164 cookie 94151402526832 notifier 4706 bl.length()=169 2019-12-08T19:36:19.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.652+0000 7fd3c5c63ec0 10 removing user index: bar.client.0 2019-12-08T19:36:19.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.652+0000 7fd3c5c63ec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-12-08T19:36:19.655 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.652+0000 7fd3c5c63ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-12-08T19:36:19.673 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:19.671+0000 7fd393fff700 10 RGWWatcher::handle_notify() notify_id 575525617725 cookie 94151402527744 notifier 4706 bl.length()=161 2019-12-08T19:36:20.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.189+0000 7fd3c5c63ec0 20 remove_watcher() i=0 2019-12-08T19:36:20.191 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.189+0000 7fd3c5c63ec0 2 removed watcher, disabling cache 2019-12-08T19:36:20.230 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.227+0000 7fd3c5c63ec0 20 remove_watcher() i=1 2019-12-08T19:36:20.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.261+0000 7fd3c5c63ec0 20 remove_watcher() i=2 2019-12-08T19:36:20.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.340+0000 7fd3c5c63ec0 20 remove_watcher() i=3 2019-12-08T19:36:20.386 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.383+0000 7fd3c5c63ec0 20 remove_watcher() i=4 2019-12-08T19:36:20.428 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.426+0000 7fd3c5c63ec0 20 remove_watcher() i=5 2019-12-08T19:36:20.478 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.476+0000 7fd3c5c63ec0 20 remove_watcher() i=6 2019-12-08T19:36:20.523 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.520+0000 7fd3c5c63ec0 20 remove_watcher() i=7 2019-12-08T19:36:20.537 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-12-08T19:36:20.661 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.659+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.683+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.686 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.683+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.693+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.695 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.693+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.697+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.700 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.697+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.705+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:20.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.872+0000 7fe78276cec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T19:36:20.874 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.872+0000 7fe78276cec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T19:36:20.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.872+0000 7fe78276cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T19:36:20.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.872+0000 7fe78276cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T19:36:20.875 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.872+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.880+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:20.883 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.880+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.887+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:20.890 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.887+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.896+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:20.898 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.896+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.917 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.914+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:20.917 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.914+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.921 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.919+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.919+0000 7fe78276cec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T19:36:20.922 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.919+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.928 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.926+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.928 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.926+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.940+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:20.943 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.940+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.960+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:20.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.960+0000 7fe78276cec0 20 zonegroup default 2019-12-08T19:36:20.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.960+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.967+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.967+0000 7fe78276cec0 10 Cannot find current period zone using local zone 2019-12-08T19:36:20.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.967+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.974+0000 7fe78276cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:20.977 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.974+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.986 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.983+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:20.986 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.983+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:20.994 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.992+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:20.995 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.992+0000 7fe78276cec0 20 zone default found 2019-12-08T19:36:20.995 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.992+0000 7fe78276cec0 20 started sync module instance, tier type = 2019-12-08T19:36:20.995 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:20.992+0000 7fe78276cec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T19:36:21.384 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=0 2019-12-08T19:36:21.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=1 2019-12-08T19:36:21.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=2 2019-12-08T19:36:21.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=3 2019-12-08T19:36:21.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=4 2019-12-08T19:36:21.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=5 2019-12-08T19:36:21.385 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.382+0000 7fe78276cec0 20 add_watcher() i=6 2019-12-08T19:36:21.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.412+0000 7fe78276cec0 20 add_watcher() i=7 2019-12-08T19:36:21.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.412+0000 7fe78276cec0 2 all 8 watchers are set, enabling cache 2019-12-08T19:36:21.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.412+0000 7fe6c3fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T19:36:21.422 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.420+0000 7fe76f195700 0 -- 172.21.15.129:0/2164014522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5649e3cbd8d0 msgr2=0x5649e3cbfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-08T19:36:21.470 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:21.467+0000 7fe770197700 0 -- 172.21.15.129:0/2744663459 >> [v2:172.21.15.129:6800/9246,v1:172.21.15.129:6801/9246] conn(0x7fe6b0022490 msgr2=0x7fe6b0024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:22.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.857+0000 7fe6ae7fc700 20 reqs_thread_entry: start 2019-12-08T19:36:22.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.857+0000 7fe6a7fff700 20 reqs_thread_entry: start 2019-12-08T19:36:22.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.857+0000 7fe78276cec0 20 init_complete bucket index max shards: 0 2019-12-08T19:36:22.860 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.858+0000 7fe6a67fc700 20 reqs_thread_entry: start 2019-12-08T19:36:22.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.858+0000 7fe78276cec0 20 get_system_obj_state: rctx=0x7ffd2a879148 obj=default.rgw.meta:users.uid:foo.client.0 state=0x5649e3d3b580 s->prefetch_data=0 2019-12-08T19:36:22.861 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.858+0000 7fe78276cec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-08T19:36:22.881 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.879+0000 7fe78276cec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-08T19:36:22.881 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.879+0000 7fe78276cec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:22.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.879+0000 7fe78276cec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T19:36:22.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.879+0000 7fe78276cec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T19:36:22.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.879+0000 7fe78276cec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T19:36:22.882 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.879+0000 7fe78276cec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:22.894 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.891+0000 7fe78276cec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-08T19:36:22.894 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.891+0000 7fe78276cec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-08T19:36:22.894 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.891+0000 7fe78276cec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:22.910 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.908+0000 7fe78276cec0 10 removing key index: YZGSEIIYRKKCPAVLVUEL 2019-12-08T19:36:22.910 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.908+0000 7fe78276cec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=170 2019-12-08T19:36:22.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:22.920+0000 7fe7657fa700 10 RGWWatcher::handle_notify() notify_id 575525617744 cookie 94875354307088 notifier 4715 bl.length()=170 2019-12-08T19:36:23.100 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.098+0000 7fe78276cec0 10 removing email index: foo.client.0+test@test.test 2019-12-08T19:36:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.098+0000 7fe78276cec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-12-08T19:36:23.115 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.113+0000 7fe7657fa700 10 RGWWatcher::handle_notify() notify_id 575525617745 cookie 94875354307088 notifier 4715 bl.length()=178 2019-12-08T19:36:23.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.365+0000 7fe78276cec0 10 removing user buckets index 2019-12-08T19:36:23.368 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.365+0000 7fe78276cec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-12-08T19:36:23.380 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.378+0000 7fe7657fa700 10 RGWWatcher::handle_notify() notify_id 575525617726 cookie 94875354267792 notifier 4715 bl.length()=169 2019-12-08T19:36:23.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.844+0000 7fe78276cec0 10 removing user index: foo.client.0 2019-12-08T19:36:23.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.844+0000 7fe78276cec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-12-08T19:36:23.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.844+0000 7fe78276cec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-12-08T19:36:23.866 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:23.863+0000 7fe7657fa700 10 RGWWatcher::handle_notify() notify_id 575525618178 cookie 94875354265824 notifier 4715 bl.length()=161 2019-12-08T19:36:24.224 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.221+0000 7fe78276cec0 20 remove_watcher() i=0 2019-12-08T19:36:24.224 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.221+0000 7fe78276cec0 2 removed watcher, disabling cache 2019-12-08T19:36:24.294 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.291+0000 7fe78276cec0 20 remove_watcher() i=1 2019-12-08T19:36:24.342 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.340+0000 7fe78276cec0 20 remove_watcher() i=2 2019-12-08T19:36:24.541 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.539+0000 7fe78276cec0 20 remove_watcher() i=3 2019-12-08T19:36:24.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.632+0000 7fe78276cec0 20 remove_watcher() i=4 2019-12-08T19:36:24.692 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.688+0000 7fe78276cec0 20 remove_watcher() i=5 2019-12-08T19:36:24.746 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.743+0000 7fe78276cec0 20 remove_watcher() i=6 2019-12-08T19:36:24.794 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.790+0000 7fe78276cec0 20 remove_watcher() i=7 2019-12-08T19:36:24.802 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-12-08T19:36:24.927 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.924+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:24.954 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.950+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:24.954 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.950+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:24.961 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.958+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:24.962 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.958+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:24.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.963+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:24.966 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.963+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:24.974 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:24.971+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:25.149 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.145+0000 7fb965876ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T19:36:25.149 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.145+0000 7fb965876ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T19:36:25.149 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.145+0000 7fb965876ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T19:36:25.150 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.145+0000 7fb965876ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T19:36:25.150 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.145+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.160 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.157+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:25.161 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.157+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.170 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.167+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:25.171 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.167+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.179 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.176+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:25.180 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.176+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.186+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:25.189 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.186+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.200 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.196+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:25.200 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.196+0000 7fb965876ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T19:36:25.200 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.196+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.202+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:25.205 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.202+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.219 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.215+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:25.219 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.215+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.226 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.222+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:25.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.222+0000 7fb965876ec0 20 zonegroup default 2019-12-08T19:36:25.227 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.222+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.230+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.230+0000 7fb965876ec0 10 Cannot find current period zone using local zone 2019-12-08T19:36:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.230+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.239 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.235+0000 7fb965876ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:25.239 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.235+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.246 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.243+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:25.247 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.243+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:25.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.250+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:25.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.250+0000 7fb965876ec0 20 zone default found 2019-12-08T19:36:25.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.250+0000 7fb965876ec0 20 started sync module instance, tier type = 2019-12-08T19:36:25.254 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.250+0000 7fb965876ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T19:36:25.593 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.590+0000 7fb952aa0700 0 -- 172.21.15.129:0/371025343 >> [v2:172.21.15.181:6806/10375,v1:172.21.15.181:6809/10375] conn(0x55c0c1726bf0 msgr2=0x55c0c1729010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:25.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=0 2019-12-08T19:36:25.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=1 2019-12-08T19:36:25.650 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=2 2019-12-08T19:36:25.651 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=3 2019-12-08T19:36:25.651 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=4 2019-12-08T19:36:25.651 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=5 2019-12-08T19:36:25.651 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.646+0000 7fb965876ec0 20 add_watcher() i=6 2019-12-08T19:36:25.681 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.678+0000 7fb965876ec0 20 add_watcher() i=7 2019-12-08T19:36:25.682 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.678+0000 7fb965876ec0 2 all 8 watchers are set, enabling cache 2019-12-08T19:36:25.682 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.678+0000 7fb9237de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T19:36:25.698 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:25.694+0000 7fb952aa0700 0 -- 172.21.15.129:0/3517618503 >> v2:172.21.15.181:3300/0 conn(0x55c0c1739bd0 msgr2=0x55c0c173e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:27.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.259+0000 7fb891ffb700 20 reqs_thread_entry: start 2019-12-08T19:36:27.263 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.260+0000 7fb88b7fe700 20 reqs_thread_entry: start 2019-12-08T19:36:27.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.260+0000 7fb965876ec0 20 init_complete bucket index max shards: 0 2019-12-08T19:36:27.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.260+0000 7fb889ffb700 20 reqs_thread_entry: start 2019-12-08T19:36:27.264 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.260+0000 7fb965876ec0 20 get_system_obj_state: rctx=0x7fff84279f98 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55c0c17b7fc0 s->prefetch_data=0 2019-12-08T19:36:27.265 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.260+0000 7fb965876ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-08T19:36:27.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.270+0000 7fb965876ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-08T19:36:27.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.270+0000 7fb965876ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T19:36:27.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.270+0000 7fb965876ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-08T19:36:27.273 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.270+0000 7fb965876ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-08T19:36:27.274 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.270+0000 7fb965876ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-08T19:36:27.274 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.270+0000 7fb965876ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:27.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.280+0000 7fb965876ec0 20 rados_obj.operate() r=0 bl.length=505 2019-12-08T19:36:27.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.280+0000 7fb965876ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-08T19:36:27.284 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.280+0000 7fb965876ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-08T19:36:27.293 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.290+0000 7fb965876ec0 10 removing key index: TUAVGJGNDIUFSGEVIIJK 2019-12-08T19:36:27.294 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.290+0000 7fb965876ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=170 2019-12-08T19:36:27.301 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.297+0000 7fb933fff700 10 RGWWatcher::handle_notify() notify_id 575525617727 cookie 94286367541664 notifier 4727 bl.length()=170 2019-12-08T19:36:27.772 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.768+0000 7fb965876ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-12-08T19:36:27.772 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.768+0000 7fb965876ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-12-08T19:36:27.786 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.783+0000 7fb933fff700 10 RGWWatcher::handle_notify() notify_id 575525617746 cookie 94286367583216 notifier 4727 bl.length()=193 2019-12-08T19:36:27.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.945+0000 7fb965876ec0 10 removing user buckets index 2019-12-08T19:36:27.949 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.945+0000 7fb965876ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-12-08T19:36:27.968 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:27.965+0000 7fb933fff700 10 RGWWatcher::handle_notify() notify_id 575525617747 cookie 94286367580736 notifier 4727 bl.length()=184 2019-12-08T19:36:28.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:28.413+0000 7fb965876ec0 10 removing user index: testx$tenanteduser.client.0 2019-12-08T19:36:28.417 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:28.414+0000 7fb965876ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-12-08T19:36:28.418 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:28.414+0000 7fb965876ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-12-08T19:36:28.436 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:28.427+0000 7fb933fff700 10 RGWWatcher::handle_notify() notify_id 575525618165 cookie 94286367577056 notifier 4727 bl.length()=176 2019-12-08T19:36:29.062 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.057+0000 7fb965876ec0 20 remove_watcher() i=0 2019-12-08T19:36:29.064 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.057+0000 7fb965876ec0 2 removed watcher, disabling cache 2019-12-08T19:36:29.132 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.129+0000 7fb965876ec0 20 remove_watcher() i=1 2019-12-08T19:36:29.167 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.164+0000 7fb965876ec0 20 remove_watcher() i=2 2019-12-08T19:36:29.217 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.213+0000 7fb965876ec0 20 remove_watcher() i=3 2019-12-08T19:36:29.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.320+0000 7fb965876ec0 20 remove_watcher() i=4 2019-12-08T19:36:29.367 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.363+0000 7fb965876ec0 20 remove_watcher() i=5 2019-12-08T19:36:29.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.411+0000 7fb965876ec0 20 remove_watcher() i=6 2019-12-08T19:36:29.465 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:29.462+0000 7fb965876ec0 20 remove_watcher() i=7 2019-12-08T19:36:29.479 INFO:tasks.s3tests:Removing s3-tests... 2019-12-08T19:36:29.479 INFO:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-12-08T19:36:29.644 INFO:teuthology.run_tasks:Running task swift... 2019-12-08T19:36:29.666 INFO:tasks.swift:clients=['client.0'] 2019-12-08T19:36:29.666 INFO:tasks.swift:Downloading swift... 2019-12-08T19:36:29.666 INFO:teuthology.orchestra.run.smithi129:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-12-08T19:36:29.768 INFO:teuthology.orchestra.run.smithi129.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-12-08T19:36:30.136 INFO:tasks.swift:Creating rgw users... 2019-12-08T19:36:30.137 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret mquZOmUnpaLurNpw9sii5JIZYHZaTRJ0e1X5moVX2ZWNq+0ceJVVag== --email foo.client.0+test@test.test --key-type swift --access full 2019-12-08T19:36:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.270+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.294+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.297 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.294+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.302+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.305 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.302+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.306+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.309 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.306+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.316 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.313+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:30.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.497+0000 7f324e182ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T19:36:30.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.497+0000 7f324e182ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T19:36:30.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.497+0000 7f324e182ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T19:36:30.501 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.497+0000 7f324e182ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T19:36:30.502 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.497+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.508+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:30.512 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.508+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.520 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.517+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:30.520 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.517+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.537 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.534+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:30.537 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.534+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.557 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.553+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:30.557 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.553+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.558+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.558+0000 7f324e182ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T19:36:30.562 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.558+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.566 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.563+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.566 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.563+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.576 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.572+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:30.576 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.572+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.588+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:30.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.588+0000 7f324e182ec0 20 zonegroup default 2019-12-08T19:36:30.591 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.588+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.592+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.595 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.592+0000 7f324e182ec0 10 Cannot find current period zone using local zone 2019-12-08T19:36:30.596 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.592+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.600 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.597+0000 7f324e182ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:30.600 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.597+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.604+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:30.607 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.604+0000 7f324e182ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:30.615 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.611+0000 7f324e182ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:30.615 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.611+0000 7f324e182ec0 20 zone default found 2019-12-08T19:36:30.615 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.611+0000 7f324e182ec0 20 started sync module instance, tier type = 2019-12-08T19:36:30.615 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:30.611+0000 7f324e182ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T19:36:31.014 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.010+0000 7f324e182ec0 20 add_watcher() i=0 2019-12-08T19:36:31.014 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.010+0000 7f324e182ec0 20 add_watcher() i=1 2019-12-08T19:36:31.014 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.010+0000 7f324e182ec0 20 add_watcher() i=2 2019-12-08T19:36:31.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.010+0000 7f324e182ec0 20 add_watcher() i=3 2019-12-08T19:36:31.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.010+0000 7f324e182ec0 20 add_watcher() i=4 2019-12-08T19:36:31.015 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.010+0000 7f324e182ec0 20 add_watcher() i=5 2019-12-08T19:36:31.019 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.016+0000 7f324e182ec0 20 add_watcher() i=6 2019-12-08T19:36:31.068 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.064+0000 7f324e182ec0 20 add_watcher() i=7 2019-12-08T19:36:31.068 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.064+0000 7f324e182ec0 2 all 8 watchers are set, enabling cache 2019-12-08T19:36:31.069 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.065+0000 7f320bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T19:36:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.096+0000 7f323b3ac700 0 -- >> v2:172.21.15.129:3300/0 conn(0x56364c64e2c0 msgr2=0x56364c6506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:31.169 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:31.165+0000 7f323bbad700 0 -- 172.21.15.129:0/3768414665 >> [v2:172.21.15.129:6802/10195,v1:172.21.15.129:6803/10195] conn(0x56364c65fef0 msgr2=0x56364c662310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:32.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.714+0000 7f317a7fc700 20 reqs_thread_entry: start 2019-12-08T19:36:32.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.715+0000 7f3173fff700 20 reqs_thread_entry: start 2019-12-08T19:36:32.717 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.715+0000 7f324e182ec0 20 init_complete bucket index max shards: 0 2019-12-08T19:36:32.718 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.715+0000 7f31727fc700 20 reqs_thread_entry: start 2019-12-08T19:36:32.718 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.715+0000 7f324e182ec0 20 get_system_obj_state: rctx=0x7ffcccbea448 obj=default.rgw.meta:users.uid:foo.client.0 state=0x56364c6c6da0 s->prefetch_data=0 2019-12-08T19:36:32.718 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.715+0000 7f324e182ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-08T19:36:32.727 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.725+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-08T19:36:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.725+0000 7f324e182ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.725+0000 7f324e182ec0 20 get_system_obj_state: rctx=0x7ffcccbea458 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x56364c6c8b30 s->prefetch_data=0 2019-12-08T19:36:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.725+0000 7f324e182ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-08T19:36:32.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.734+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-08T19:36:32.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.734+0000 7f324e182ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-08T19:36:32.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.734+0000 7f324e182ec0 20 get_system_obj_state: rctx=0x7ffcccbea068 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x56364c6c8b30 s->prefetch_data=0 2019-12-08T19:36:32.737 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.734+0000 7f324e182ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-12-08T19:36:32.743 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.741+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-08T19:36:32.743 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.741+0000 7f324e182ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-08T19:36:32.744 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.741+0000 7f324e182ec0 20 get_system_obj_state: rctx=0x7ffcccbea1e8 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x56364c6c8de0 s->prefetch_data=0 2019-12-08T19:36:32.744 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.741+0000 7f324e182ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-12-08T19:36:32.750 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.748+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-08T19:36:32.751 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.748+0000 7f324e182ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-08T19:36:32.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.847+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T19:36:32.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.847+0000 7f324e182ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:32.850 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.847+0000 7f324e182ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-12-08T19:36:32.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.862+0000 7f3220ff9700 10 RGWWatcher::handle_notify() notify_id 575525618179 cookie 94791209765776 notifier 4739 bl.length()=612 2019-12-08T19:36:32.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.862+0000 7f3220ff9700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T19:36:32.865 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:32.862+0000 7f3220ff9700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.008+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-08T19:36:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.009+0000 7f324e182ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-08T19:36:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.009+0000 7f324e182ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-08T19:36:33.028 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.026+0000 7f3220ff9700 10 RGWWatcher::handle_notify() notify_id 575525617748 cookie 94791209794016 notifier 4739 bl.length()=194 2019-12-08T19:36:33.029 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.026+0000 7f3220ff9700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-08T19:36:33.029 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.026+0000 7f3220ff9700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-08T19:36:33.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.333+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-08T19:36:33.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.333+0000 7f324e182ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-08T19:36:33.336 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.333+0000 7f324e182ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-12-08T19:36:33.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.347+0000 7f3220ff9700 10 RGWWatcher::handle_notify() notify_id 575525617728 cookie 94791209767824 notifier 4739 bl.length()=183 2019-12-08T19:36:33.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.348+0000 7f3220ff9700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-08T19:36:33.350 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.348+0000 7f3220ff9700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-08T19:36:33.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.962+0000 7f324e182ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T19:36:33.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.962+0000 7f324e182ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:33.965 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.962+0000 7f324e182ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-12-08T19:36:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.981+0000 7f3220ff9700 10 RGWWatcher::handle_notify() notify_id 575525618180 cookie 94791209765776 notifier 4739 bl.length()=636 2019-12-08T19:36:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.981+0000 7f3220ff9700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-08T19:36:33.984 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:33.981+0000 7f3220ff9700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-08T19:36:34.393 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T19:36:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "foo.client.0", 2019-12-08T19:36:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. foo.client.0 foo", 2019-12-08T19:36:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "foo.client.0+test@test.test", 2019-12-08T19:36:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [ 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "id": "foo.client.0:foo", 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "permissions": "full-control" 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T19:36:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [], 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [ 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "foo.client.0:foo", 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "mquZOmUnpaLurNpw9sii5JIZYHZaTRJ0e1X5moVX2ZWNq+0ceJVVag==" 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T19:36:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T19:36:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T19:36:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T19:36:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T19:36:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T19:36:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T19:36:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T19:36:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T19:36:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T19:36:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T19:36:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T19:36:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T19:36:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T19:36:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T19:36:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T19:36:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T19:36:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T19:36:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T19:36:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T19:36:34.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.451+0000 7f324e182ec0 20 remove_watcher() i=0 2019-12-08T19:36:34.454 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.451+0000 7f324e182ec0 2 removed watcher, disabling cache 2019-12-08T19:36:34.514 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.511+0000 7f324e182ec0 20 remove_watcher() i=1 2019-12-08T19:36:34.561 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.558+0000 7f324e182ec0 20 remove_watcher() i=2 2019-12-08T19:36:34.612 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.610+0000 7f324e182ec0 20 remove_watcher() i=3 2019-12-08T19:36:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.661+0000 7f324e182ec0 20 remove_watcher() i=4 2019-12-08T19:36:34.723 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.720+0000 7f324e182ec0 20 remove_watcher() i=5 2019-12-08T19:36:34.787 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.784+0000 7f324e182ec0 20 remove_watcher() i=6 2019-12-08T19:36:34.847 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:34.844+0000 7f324e182ec0 20 remove_watcher() i=7 2019-12-08T19:36:34.865 INFO:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret SFrQsyrM1560/GBdVGdVUqraC99mHt6WCFqj5iwCdPUBgGLgtOXWsQ== --email bar.client.0+test@test.test --key-type swift --access full 2019-12-08T19:36:35.093 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.090+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.141+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.141+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.150+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.152 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.150+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.156+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.159 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.156+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.165+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:35.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.307+0000 7f9009928ec0 20 RGWRados::pool_iterate: got zone_info.9161132f-da48-40b4-a853-f341a1971d48 2019-12-08T19:36:35.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.307+0000 7f9009928ec0 20 RGWRados::pool_iterate: got zonegroup_info.8b2e02d0-23d0-407d-9a59-4c0f854ba904 2019-12-08T19:36:35.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.307+0000 7f9009928ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-08T19:36:35.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.307+0000 7f9009928ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-08T19:36:35.310 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.307+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.322+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.322+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.333+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:35.335 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.333+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.346 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.344+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:35.347 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.344+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.366 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.364+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:35.366 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.364+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.369+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.369+0000 7f9009928ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-08T19:36:35.371 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.369+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.378 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.376+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.379 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.376+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.388+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:35.390 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.388+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.401+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-08T19:36:35.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.401+0000 7f9009928ec0 20 zonegroup default 2019-12-08T19:36:35.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.401+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.414 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.412+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.414 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.412+0000 7f9009928ec0 10 Cannot find current period zone using local zone 2019-12-08T19:36:35.414 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.412+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.425 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.419+0000 7f9009928ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-08T19:36:35.425 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.419+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.433 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.431+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-08T19:36:35.434 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.431+0000 7f9009928ec0 20 rados->read ofs=0 len=0 2019-12-08T19:36:35.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.448+0000 7f9009928ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-08T19:36:35.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.448+0000 7f9009928ec0 20 zone default found 2019-12-08T19:36:35.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.448+0000 7f9009928ec0 20 started sync module instance, tier type = 2019-12-08T19:36:35.451 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.448+0000 7f9009928ec0 20 started zone id=9161132f-da48-40b4-a853-f341a1971d48 (name=default) with tier type = 2019-12-08T19:36:35.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=0 2019-12-08T19:36:35.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=1 2019-12-08T19:36:35.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=2 2019-12-08T19:36:35.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=3 2019-12-08T19:36:35.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=4 2019-12-08T19:36:35.863 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=5 2019-12-08T19:36:35.864 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.860+0000 7f9009928ec0 20 add_watcher() i=6 2019-12-08T19:36:35.923 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.921+0000 7f9009928ec0 20 add_watcher() i=7 2019-12-08T19:36:35.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.921+0000 7f9009928ec0 2 all 8 watchers are set, enabling cache 2019-12-08T19:36:35.924 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.921+0000 7f8fc77de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-08T19:36:35.941 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.938+0000 7f8ff6351700 0 -- 172.21.15.129:0/679442098 >> [v2:172.21.15.129:3301/0,v1:172.21.15.129:6790/0] conn(0x555949087e00 msgr2=0x55594908a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:35.964 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:35.962+0000 7f8ff7353700 0 -- 172.21.15.129:0/1704425208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555949086a30 msgr2=0x555949086e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-08T19:36:37.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.631+0000 7f8f35ffb700 20 reqs_thread_entry: start 2019-12-08T19:36:37.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.632+0000 7f8f2f7fe700 20 reqs_thread_entry: start 2019-12-08T19:36:37.634 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.632+0000 7f9009928ec0 20 init_complete bucket index max shards: 0 2019-12-08T19:36:37.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.632+0000 7f8f2dffb700 20 reqs_thread_entry: start 2019-12-08T19:36:37.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.632+0000 7f9009928ec0 20 get_system_obj_state: rctx=0x7fff8e80d978 obj=default.rgw.meta:users.uid:bar.client.0 state=0x5559490fe030 s->prefetch_data=0 2019-12-08T19:36:37.635 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.632+0000 7f9009928ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-08T19:36:37.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.640+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-08T19:36:37.642 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.640+0000 7f9009928ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:37.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.640+0000 7f9009928ec0 20 get_system_obj_state: rctx=0x7fff8e80d988 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x5559490ffdc0 s->prefetch_data=0 2019-12-08T19:36:37.643 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.640+0000 7f9009928ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-08T19:36:37.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.646+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-08T19:36:37.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.646+0000 7f9009928ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-08T19:36:37.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.646+0000 7f9009928ec0 20 get_system_obj_state: rctx=0x7fff8e80d598 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x5559490ffdc0 s->prefetch_data=0 2019-12-08T19:36:37.649 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.646+0000 7f9009928ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-12-08T19:36:37.656 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.654+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-08T19:36:37.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.654+0000 7f9009928ec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-08T19:36:37.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.654+0000 7f9009928ec0 20 get_system_obj_state: rctx=0x7fff8e80d718 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x555949100070 s->prefetch_data=0 2019-12-08T19:36:37.657 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.654+0000 7f9009928ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-12-08T19:36:37.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.660+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-08T19:36:37.662 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.660+0000 7f9009928ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-08T19:36:37.707 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.705+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T19:36:37.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.705+0000 7f9009928ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:37.708 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.705+0000 7f9009928ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-12-08T19:36:37.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.717+0000 7f8fd7fff700 10 RGWWatcher::handle_notify() notify_id 575525617729 cookie 93841965624096 notifier 4996 bl.length()=612 2019-12-08T19:36:37.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.717+0000 7f8fd7fff700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T19:36:37.719 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.717+0000 7f8fd7fff700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:37.818 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.816+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-08T19:36:37.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.816+0000 7f9009928ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-08T19:36:37.819 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.816+0000 7f9009928ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-08T19:36:37.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.823+0000 7f8fd7fff700 10 RGWWatcher::handle_notify() notify_id 575525617730 cookie 93841965624096 notifier 4996 bl.length()=194 2019-12-08T19:36:37.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.823+0000 7f8fd7fff700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-08T19:36:37.826 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.823+0000 7f8fd7fff700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-08T19:36:37.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.915+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-08T19:36:37.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.915+0000 7f9009928ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-08T19:36:37.918 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.915+0000 7f9009928ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-12-08T19:36:37.945 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.943+0000 7f8fd7fff700 10 RGWWatcher::handle_notify() notify_id 575525618166 cookie 93841965629120 notifier 4996 bl.length()=183 2019-12-08T19:36:37.946 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.943+0000 7f8fd7fff700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-08T19:36:37.946 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:37.943+0000 7f8fd7fff700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-08T19:36:38.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:38.400+0000 7f9009928ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T19:36:38.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:38.400+0000 7f9009928ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:38.403 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:38.400+0000 7f9009928ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-12-08T19:36:38.414 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:38.412+0000 7f8fd7fff700 10 RGWWatcher::handle_notify() notify_id 575525617731 cookie 93841965624096 notifier 4996 bl.length()=636 2019-12-08T19:36:38.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:38.412+0000 7f8fd7fff700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-08T19:36:38.415 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:38.412+0000 7f8fd7fff700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-08T19:36:38.992 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2019-12-08T19:36:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "user_id": "bar.client.0", 2019-12-08T19:36:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "display_name": "Mr. bar.client.0 bar", 2019-12-08T19:36:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "email": "bar.client.0+test@test.test", 2019-12-08T19:36:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "suspended": 0, 2019-12-08T19:36:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max_buckets": 1000, 2019-12-08T19:36:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "subusers": [ 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "id": "bar.client.0:bar", 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "permissions": "full-control" 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "keys": [], 2019-12-08T19:36:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "swift_keys": [ 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "user": "bar.client.0:bar", 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "secret_key": "SFrQsyrM1560/GBdVGdVUqraC99mHt6WCFqj5iwCdPUBgGLgtOXWsQ==" 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": [], 2019-12-08T19:36:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "op_mask": "read, write, delete", 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "default_placement": "", 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "default_storage_class": "", 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "placement_tags": [], 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "bucket_quota": { 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T19:36:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "user_quota": { 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": false, 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "check_on_raw": false, 2019-12-08T19:36:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size": -1, 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "max_size_kb": 0, 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "max_objects": -1 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "temp_url_keys": [], 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "rgw", 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "mfa_ids": [] 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout:} 2019-12-08T19:36:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T19:36:39.054 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.051+0000 7f9009928ec0 20 remove_watcher() i=0 2019-12-08T19:36:39.054 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.051+0000 7f9009928ec0 2 removed watcher, disabling cache 2019-12-08T19:36:39.093 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.089+0000 7f9009928ec0 20 remove_watcher() i=1 2019-12-08T19:36:39.128 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.125+0000 7f9009928ec0 20 remove_watcher() i=2 2019-12-08T19:36:39.167 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.165+0000 7f9009928ec0 20 remove_watcher() i=3 2019-12-08T19:36:39.201 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.199+0000 7f9009928ec0 20 remove_watcher() i=4 2019-12-08T19:36:39.242 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.239+0000 7f9009928ec0 20 remove_watcher() i=5 2019-12-08T19:36:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.290+0000 7f9009928ec0 20 remove_watcher() i=6 2019-12-08T19:36:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:2019-12-08T19:36:39.335+0000 7f9009928ec0 20 remove_watcher() i=7 2019-12-08T19:36:39.347 INFO:tasks.swift:Configuring testswift... 2019-12-08T19:36:39.347 INFO:teuthology.orchestra.run.smithi129:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-12-08T19:36:41.861 INFO:teuthology.orchestra.run.smithi129.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-12-08T19:36:43.187 INFO:teuthology.orchestra.run.smithi129.stdout:Installing setuptools, pip, wheel...done. 2019-12-08T19:36:43.446 INFO:teuthology.orchestra.run.smithi129.stderr:Cache entry deserialization failed, entry ignored 2019-12-08T19:36:43.573 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting pip 2019-12-08T19:36:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-08T19:36:43.712 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: pip 2019-12-08T19:36:43.712 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: pip 9.0.1 2019-12-08T19:36:43.771 INFO:teuthology.orchestra.run.smithi129.stdout: Uninstalling pip-9.0.1: 2019-12-08T19:36:43.811 INFO:teuthology.orchestra.run.smithi129.stdout: Successfully uninstalled pip-9.0.1 2019-12-08T19:36:44.400 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed pip-19.3.1 2019-12-08T19:36:44.673 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T19:36:45.212 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting setuptools==32.3.1 2019-12-08T19:36:45.215 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-08T19:36:45.257 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: setuptools 2019-12-08T19:36:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: Found existing installation: setuptools 28.8.0 2019-12-08T19:36:45.272 INFO:teuthology.orchestra.run.smithi129.stdout: Uninstalling setuptools-28.8.0: 2019-12-08T19:36:45.281 INFO:teuthology.orchestra.run.smithi129.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-08T19:36:45.410 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed setuptools-32.3.1 2019-12-08T19:36:45.848 INFO:teuthology.orchestra.run.smithi129.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-08T19:36:46.034 INFO:teuthology.orchestra.run.smithi129.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-08T19:36:46.096 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting nose>=1.0.0 2019-12-08T19:36:46.100 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-08T19:36:46.382 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting boto>=2.0b4 2019-12-08T19:36:46.390 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-08T19:36:46.532 INFO:teuthology.orchestra.run.smithi129.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-08T19:36:46.799 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting gevent>=0.14 2019-12-08T19:36:46.812 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T19:36:47.062 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting isodate>=0.4.4 2019-12-08T19:36:47.064 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-12-08T19:36:47.143 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting eventlet>=0.9.14 2019-12-08T19:36:47.181 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-12-08T19:36:47.348 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting netifaces>=0.5 2019-12-08T19:36:47.358 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T19:36:47.448 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting xattr<0.7.0,>=0.6 2019-12-08T19:36:47.457 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-12-08T19:36:47.892 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting simplejson>=2.1.3 2019-12-08T19:36:47.902 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-12-08T19:36:48.282 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting WebOb==1.0 2019-12-08T19:36:48.295 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-12-08T19:36:48.806 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-08T19:36:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-08T19:36:48.879 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting six 2019-12-08T19:36:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-08T19:36:48.945 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting monotonic>=1.4 2019-12-08T19:36:48.956 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-12-08T19:36:49.046 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting dnspython>=1.15.0 2019-12-08T19:36:49.057 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-12-08T19:36:49.196 INFO:teuthology.orchestra.run.smithi129.stdout:Collecting enum34; python_version < "3.4" 2019-12-08T19:36:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-12-08T19:36:49.241 INFO:teuthology.orchestra.run.smithi129.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-12-08T19:36:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for xattr (setup.py): started 2019-12-08T19:36:49.666 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-12-08T19:36:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24120 sha256=f3690841eb7d148f4a8b36a96f1d6f415b5612b090a9af3e566ae744406c0e94 2019-12-08T19:36:49.667 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-12-08T19:36:49.670 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for simplejson (setup.py): started 2019-12-08T19:36:50.585 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-12-08T19:36:50.587 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113124 sha256=a74fba7c740af1407009331f7737ca3fb6f798ef65cf970beb3b0b339fb84df8 2019-12-08T19:36:50.587 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-12-08T19:36:50.607 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for WebOb (setup.py): started 2019-12-08T19:36:50.839 INFO:teuthology.orchestra.run.smithi129.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-12-08T19:36:50.839 INFO:teuthology.orchestra.run.smithi129.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=48200f6dae3cb9fe5291abd36e1a4bc9e61f8fafe330e5c60225ef2b789224ab 2019-12-08T19:36:50.839 INFO:teuthology.orchestra.run.smithi129.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-12-08T19:36:50.849 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully built xattr simplejson WebOb 2019-12-08T19:36:50.899 INFO:teuthology.orchestra.run.smithi129.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-12-08T19:36:52.379 INFO:teuthology.orchestra.run.smithi129.stdout:Successfully installed PyYAML-5.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-12-08T19:36:52.650 INFO:teuthology.orchestra.run.smithi129.stdout:running develop 2019-12-08T19:36:52.698 INFO:teuthology.orchestra.run.smithi129.stdout:running egg_info 2019-12-08T19:36:52.699 INFO:teuthology.orchestra.run.smithi129.stdout:creating swift.egg-info 2019-12-08T19:36:52.699 INFO:teuthology.orchestra.run.smithi129.stdout:writing swift.egg-info/PKG-INFO 2019-12-08T19:36:52.699 INFO:teuthology.orchestra.run.smithi129.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-12-08T19:36:52.699 INFO:teuthology.orchestra.run.smithi129.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-12-08T19:36:52.703 INFO:teuthology.orchestra.run.smithi129.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-12-08T19:36:52.703 INFO:teuthology.orchestra.run.smithi129.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-08T19:36:52.706 INFO:teuthology.orchestra.run.smithi129.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-12-08T19:36:52.707 INFO:teuthology.orchestra.run.smithi129.stdout:reading manifest template 'MANIFEST.in' 2019-12-08T19:36:52.707 INFO:teuthology.orchestra.run.smithi129.stderr:warning: no files found matching 'ChangeLog' 2019-12-08T19:36:52.711 INFO:teuthology.orchestra.run.smithi129.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-08T19:36:52.712 INFO:teuthology.orchestra.run.smithi129.stdout:running build_ext 2019-12-08T19:36:52.712 INFO:teuthology.orchestra.run.smithi129.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-12-08T19:36:52.712 INFO:teuthology.orchestra.run.smithi129.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-12-08T19:36:52.729 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.730 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.732 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.733 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.734 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.735 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.735 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.736 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.737 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.738 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.739 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.740 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.741 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.742 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.743 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.744 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.745 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.746 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.747 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.748 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.748 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.752 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.754 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.760 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.762 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.764 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.765 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.767 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.768 INFO:teuthology.orchestra.run.smithi129.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-08T19:36:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: 2019-12-08T19:36:52.769 INFO:teuthology.orchestra.run.smithi129.stdout:Installed /home/ubuntu/cephtest/swift 2019-12-08T19:36:52.769 INFO:teuthology.orchestra.run.smithi129.stdout:Processing dependencies for swift==1.4.4 2019-12-08T19:36:52.770 INFO:teuthology.orchestra.run.smithi129.stdout:Finished processing dependencies for swift==1.4.4 2019-12-08T19:36:52.781 INFO:teuthology.orchestra.run.smithi129:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-12-08T19:36:52.828 INFO:teuthology.orchestra.run.smithi129:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-12-08T19:37:22.228 INFO:teuthology.orchestra.run.smithi129.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-12-08T19:37:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-12-08T19:39:43.292 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:56:14.719 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.301 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:56:14.727 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.308 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:56:14.734 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.313 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:56:14.739 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.314 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:56:14.907 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.326 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:56:14.919 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.460 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:56:14.888 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.482 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:56:14.907 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:39:43.569 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:56:15.150 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-12-08T19:42:23.850 INFO:teuthology.orchestra.run.smithi129.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-12-08T19:42:28.217 INFO:teuthology.orchestra.run.smithi129.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-12-08T19:42:44.161 INFO:teuthology.orchestra.run.smithi129.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-12-08T19:42:54.265 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:59:25.693 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.310 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:59:25.737 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.328 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:59:25.753 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.332 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:59:25.758 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.342 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:59:25.934 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.348 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:59:25.941 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.415 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:06:59:25.843 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.434 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:06:59:25.860 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:42:54.484 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:06:59:26.077 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:43:00.853 INFO:teuthology.orchestra.run.smithi129.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-12-08T19:43:48.627 INFO:teuthology.orchestra.run.smithi129.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-12-08T19:43:49.226 INFO:teuthology.orchestra.run.smithi129.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-12-08T19:44:51.496 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:01:22.923 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.503 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:01:22.930 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.509 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:01:22.935 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.517 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:01:22.942 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.517 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:01:23.108 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.521 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:01:23.114 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.601 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:01:23.026 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.611 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:01:23.038 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:44:51.617 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:01:23.210 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:47:17.837 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:17.830+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:18.860 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:18.856+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:19.891 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:19.888+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:20.904 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:20.900+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:21.934 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:21.930+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:22.904 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:22.899+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:23.899 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:23.895+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:24.952 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:24.948+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:25.990 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:25.986+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:27.042 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:27.038+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:28.066 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:28.061+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:29.087 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:29.082+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:30.116 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:30.111+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:31.119 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:31.114+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:32.091 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:32.086+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:33.091 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:33.087+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:34.142 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:34.138+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:35.106 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:35.101+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:36.079 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:36.075+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:37.037 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:37.031+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:38.052 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:38.047+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:39.098 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:39.093+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:40.073 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:40.067+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:41.080 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:41.071+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:42.071 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:42.067+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:43.099 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:43.094+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:43.915 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:47:43.905+0000 1694f700 -1 osd.0 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:01.103820+0000 front 2019-12-08T19:47:02.546681+0000 (oldest deadline 2019-12-08T19:47:43.888200+0000) 2019-12-08T19:47:43.928 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:47:43.918+0000 1694f700 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:46:59.719949+0000 front 2019-12-08T19:47:01.087151+0000 (oldest deadline 2019-12-08T19:47:43.619628+0000) 2019-12-08T19:47:44.064 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:44.060+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:44.238 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:47:44.226+0000 1694f700 -1 osd.2 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.550565+0000 front 2019-12-08T19:47:02.548185+0000 (oldest deadline 2019-12-08T19:47:44.039153+0000) 2019-12-08T19:47:44.933 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:47:44.927+0000 1694f700 -1 osd.0 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:01.103820+0000 front 2019-12-08T19:47:02.546681+0000 (oldest deadline 2019-12-08T19:47:43.888200+0000) 2019-12-08T19:47:44.983 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:47:44.977+0000 1694f700 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:46:59.719949+0000 front 2019-12-08T19:47:01.087151+0000 (oldest deadline 2019-12-08T19:47:43.619628+0000) 2019-12-08T19:47:45.044 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:45.039+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:45.243 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:47:45.237+0000 1694f700 -1 osd.2 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.550565+0000 front 2019-12-08T19:47:02.548185+0000 (oldest deadline 2019-12-08T19:47:44.039153+0000) 2019-12-08T19:47:45.953 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:47:45.933+0000 1694f700 -1 osd.0 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:01.103820+0000 front 2019-12-08T19:47:02.546681+0000 (oldest deadline 2019-12-08T19:47:43.888200+0000) 2019-12-08T19:47:46.017 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:47:46.011+0000 1694f700 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:46:59.719949+0000 front 2019-12-08T19:47:01.087151+0000 (oldest deadline 2019-12-08T19:47:43.619628+0000) 2019-12-08T19:47:46.021 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:46.012+0000 1694f700 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:00.303055+0000 front 2019-12-08T19:47:01.418917+0000 (oldest deadline 2019-12-08T19:47:45.863845+0000) 2019-12-08T19:47:46.022 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:46.017+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:46.222 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:47:46.217+0000 1694f700 -1 osd.2 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.550565+0000 front 2019-12-08T19:47:02.548185+0000 (oldest deadline 2019-12-08T19:47:44.039153+0000) 2019-12-08T19:47:46.546 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:47:46.535+0000 1694f700 -1 osd.1 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.548321+0000 front 2019-12-08T19:47:02.549396+0000 (oldest deadline 2019-12-08T19:47:45.732339+0000) 2019-12-08T19:47:46.749 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:47:46.739+0000 1694f700 -1 osd.4 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:01.088352+0000 front 2019-12-08T19:47:01.090089+0000 (oldest deadline 2019-12-08T19:47:46.184873+0000) 2019-12-08T19:47:46.984 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:47:46.978+0000 1694f700 -1 osd.5 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:46:59.719949+0000 front 2019-12-08T19:47:01.087151+0000 (oldest deadline 2019-12-08T19:47:43.619628+0000) 2019-12-08T19:47:47.000 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:47:46.995+0000 1694f700 -1 osd.0 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:01.103820+0000 front 2019-12-08T19:47:02.546681+0000 (oldest deadline 2019-12-08T19:47:43.888200+0000) 2019-12-08T19:47:47.055 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:47.049+0000 1694f700 -1 osd.7 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:00.303055+0000 front 2019-12-08T19:47:01.418917+0000 (oldest deadline 2019-12-08T19:47:45.863845+0000) 2019-12-08T19:47:47.057 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:47.053+0000 1694f700 -1 osd.7 134 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:47.233 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:47:47.217+0000 1694f700 -1 osd.2 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.550565+0000 front 2019-12-08T19:47:02.548185+0000 (oldest deadline 2019-12-08T19:47:44.039153+0000) 2019-12-08T19:47:47.391 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:47:47.382+0000 1694f700 -1 osd.3 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.348867+0000 front 2019-12-08T19:47:02.792949+0000 (oldest deadline 2019-12-08T19:47:46.983594+0000) 2019-12-08T19:47:47.536 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:47:47.527+0000 1694f700 -1 osd.1 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:02.548321+0000 front 2019-12-08T19:47:02.549396+0000 (oldest deadline 2019-12-08T19:47:45.732339+0000) 2019-12-08T19:47:47.755 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:47:47.750+0000 1694f700 -1 osd.4 134 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:47:01.088352+0000 front 2019-12-08T19:47:01.090089+0000 (oldest deadline 2019-12-08T19:47:46.184873+0000) 2019-12-08T19:47:48.343 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:48.310+0000 1694f700 -1 osd.7 135 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:47:49.370 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:47:49.340+0000 1694f700 -1 osd.7 136 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106654 2.1ds0 2:b92c6237:::9161132f-da48-40b4-a853-f341a1971d48.4507.601_405a845a3ee94dd790e2f26b6f6390fa:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e134) 2019-12-08T19:48:43.397 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:43.392+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:44.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:44.396+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:44.950 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:44.940+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:45.418 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:45.416+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:45.759 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:48:45.752+0000 1694f700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.813911+0000 front 2019-12-08T19:48:01.807389+0000 (oldest deadline 2019-12-08T19:48:45.239050+0000) 2019-12-08T19:48:45.937 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:45.931+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:46.210 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:48:46.204+0000 1694f700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.064107+0000 front 2019-12-08T19:48:01.052530+0000 (oldest deadline 2019-12-08T19:48:45.492640+0000) 2019-12-08T19:48:46.436 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:46.433+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:46.787 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:48:46.780+0000 1694f700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.813911+0000 front 2019-12-08T19:48:01.807389+0000 (oldest deadline 2019-12-08T19:48:45.239050+0000) 2019-12-08T19:48:46.825 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:48:46.821+0000 1694f700 -1 osd.2 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.286671+0000 front 2019-12-08T19:48:01.268299+0000 (oldest deadline 2019-12-08T19:48:45.874524+0000) 2019-12-08T19:48:46.924 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:46.918+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:47.208 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:48:47.193+0000 1694f700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.064107+0000 front 2019-12-08T19:48:01.052530+0000 (oldest deadline 2019-12-08T19:48:45.492640+0000) 2019-12-08T19:48:47.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:47.437+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:47.751 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:48:47.745+0000 1694f700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.813911+0000 front 2019-12-08T19:48:01.807389+0000 (oldest deadline 2019-12-08T19:48:45.239050+0000) 2019-12-08T19:48:47.865 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:48:47.860+0000 1694f700 -1 osd.2 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.286671+0000 front 2019-12-08T19:48:01.268299+0000 (oldest deadline 2019-12-08T19:48:45.874524+0000) 2019-12-08T19:48:47.974 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:47.968+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:48.240 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:48:48.233+0000 1694f700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.064107+0000 front 2019-12-08T19:48:01.052530+0000 (oldest deadline 2019-12-08T19:48:45.492640+0000) 2019-12-08T19:48:48.481 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:48.478+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:48.645 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:48:48.638+0000 1694f700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:02.400205+0000 front 2019-12-08T19:48:02.348699+0000 (oldest deadline 2019-12-08T19:48:48.230894+0000) 2019-12-08T19:48:48.739 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:48:48.733+0000 1694f700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.813911+0000 front 2019-12-08T19:48:01.807389+0000 (oldest deadline 2019-12-08T19:48:45.239050+0000) 2019-12-08T19:48:48.811 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:48:48.807+0000 1694f700 -1 osd.3 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:05.042856+0000 front 2019-12-08T19:48:05.044031+0000 (oldest deadline 2019-12-08T19:48:48.508854+0000) 2019-12-08T19:48:48.895 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:48:48.890+0000 1694f700 -1 osd.2 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.286671+0000 front 2019-12-08T19:48:01.268299+0000 (oldest deadline 2019-12-08T19:48:45.874524+0000) 2019-12-08T19:48:48.932 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:48.927+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:49.259 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:48:49.252+0000 1694f700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.064107+0000 front 2019-12-08T19:48:01.052530+0000 (oldest deadline 2019-12-08T19:48:45.492640+0000) 2019-12-08T19:48:49.525 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:49.521+0000 1694f700 -1 osd.0 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:03.357190+0000 front 2019-12-08T19:48:03.273548+0000 (oldest deadline 2019-12-08T19:48:48.890982+0000) 2019-12-08T19:48:49.527 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:49.524+0000 1694f700 -1 osd.0 141 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:49.679 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:48:49.672+0000 1694f700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:02.400205+0000 front 2019-12-08T19:48:02.348699+0000 (oldest deadline 2019-12-08T19:48:48.230894+0000) 2019-12-08T19:48:49.770 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:48:49.765+0000 1694f700 -1 osd.3 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:05.042856+0000 front 2019-12-08T19:48:05.044031+0000 (oldest deadline 2019-12-08T19:48:48.508854+0000) 2019-12-08T19:48:49.794 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:48:49.778+0000 1694f700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.813911+0000 front 2019-12-08T19:48:01.807389+0000 (oldest deadline 2019-12-08T19:48:45.239050+0000) 2019-12-08T19:48:49.881 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:48:49.876+0000 1694f700 -1 osd.2 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.286671+0000 front 2019-12-08T19:48:01.268299+0000 (oldest deadline 2019-12-08T19:48:45.874524+0000) 2019-12-08T19:48:49.973 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:49.968+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:50.265 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:48:50.258+0000 1694f700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:48:01.064107+0000 front 2019-12-08T19:48:01.052530+0000 (oldest deadline 2019-12-08T19:48:45.492640+0000) 2019-12-08T19:48:50.735 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:50.673+0000 1694f700 -1 osd.0 142 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:50.957 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:48:50.953+0000 1694f700 -1 osd.6 141 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e138) 2019-12-08T19:48:51.890 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:48:51.673+0000 1694f700 -1 osd.0 143 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106798 5.5 5:a93a5511:::notify.2:head [watch ping cookie 821346576 gen 64] snapc 0=[] ondisk+write+known_if_redirected e137) 2019-12-08T19:48:52.631 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T19:48:52.612+0000 20e25700 -1 RGWWatcher::handle_error cookie 821346576 err (107) Transport endpoint is not connected 2019-12-08T19:49:21.644 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:05:53.072 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.650 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:05:53.077 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.667 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:05:53.093 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.668 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:05:53.260 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.673 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:05:53.099 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.674 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:05:53.266 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.894 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:05:53.320 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.906 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:05:53.498 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:21.929 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:05:53.356 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.772 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:05:57.200 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.774 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:05:57.202 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.803 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:05:57.229 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.804 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:05:57.232 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.811 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:05:57.237 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.855 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:05:57.281 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.889 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:05:57.481 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.891 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:05:57.484 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:25.917 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:05:57.510 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:49:55.879 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:49:55.853+0000 34f8d700 -1 osd.1 146 build_incremental_map_msg missing incremental map 146 2019-12-08T19:49:55.890 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:49:55.872+0000 11caf700 -1 osd.1 146 build_incremental_map_msg missing incremental map 146 2019-12-08T19:49:56.016 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:49:55.926+0000 34f8d700 -1 osd.1 146 build_incremental_map_msg missing incremental map 146 2019-12-08T19:49:56.025 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:49:55.944+0000 11caf700 -1 osd.1 146 build_incremental_map_msg missing incremental map 146 2019-12-08T19:49:56.467 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:49:56.463+0000 35f8f700 -1 osd.1 146 build_incremental_map_msg missing incremental map 146 2019-12-08T19:49:57.680 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:49:57.592+0000 31786700 -1 osd.1 146 build_incremental_map_msg missing incremental map 146 2019-12-08T19:50:55.901 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:50:55.896+0000 1694f700 -1 osd.1 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.553992+0000 front 2019-12-08T19:50:14.591372+0000 (oldest deadline 2019-12-08T19:50:55.588559+0000) 2019-12-08T19:50:56.480 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:50:56.475+0000 1694f700 -1 osd.7 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:11.945614+0000 front 2019-12-08T19:50:11.948572+0000 (oldest deadline 2019-12-08T19:50:56.124550+0000) 2019-12-08T19:50:56.630 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:50:56.625+0000 1694f700 -1 osd.4 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.403235+0000 front 2019-12-08T19:50:13.640471+0000 (oldest deadline 2019-12-08T19:50:55.908297+0000) 2019-12-08T19:50:56.664 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:50:56.647+0000 1694f700 -1 osd.3 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:13.248147+0000 front 2019-12-08T19:50:13.245899+0000 (oldest deadline 2019-12-08T19:50:56.547398+0000) 2019-12-08T19:50:56.876 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:50:56.871+0000 1694f700 -1 osd.1 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.553992+0000 front 2019-12-08T19:50:14.591372+0000 (oldest deadline 2019-12-08T19:50:55.588559+0000) 2019-12-08T19:50:57.014 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:50:57.009+0000 1694f700 -1 osd.2 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:12.289312+0000 front 2019-12-08T19:50:13.294992+0000 (oldest deadline 2019-12-08T19:50:56.898793+0000) 2019-12-08T19:50:57.191 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:50:57.186+0000 1694f700 -1 osd.5 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.848107+0000 front 2019-12-08T19:50:14.859242+0000 (oldest deadline 2019-12-08T19:50:56.504707+0000) 2019-12-08T19:50:57.483 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:50:57.477+0000 1694f700 -1 osd.7 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:11.945614+0000 front 2019-12-08T19:50:11.948572+0000 (oldest deadline 2019-12-08T19:50:56.124550+0000) 2019-12-08T19:50:57.610 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:50:57.604+0000 1694f700 -1 osd.4 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.403235+0000 front 2019-12-08T19:50:13.640471+0000 (oldest deadline 2019-12-08T19:50:55.908297+0000) 2019-12-08T19:50:57.692 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:50:57.688+0000 1694f700 -1 osd.3 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:13.248147+0000 front 2019-12-08T19:50:13.245899+0000 (oldest deadline 2019-12-08T19:50:56.547398+0000) 2019-12-08T19:50:57.850 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:50:57.845+0000 1694f700 -1 osd.1 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.553992+0000 front 2019-12-08T19:50:14.591372+0000 (oldest deadline 2019-12-08T19:50:55.588559+0000) 2019-12-08T19:50:57.881 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:50:57.876+0000 1694f700 -1 osd.0 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:13.299564+0000 front 2019-12-08T19:50:11.963868+0000 (oldest deadline 2019-12-08T19:50:57.233232+0000) 2019-12-08T19:50:58.039 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:50:58.035+0000 1694f700 -1 osd.2 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:12.289312+0000 front 2019-12-08T19:50:13.294992+0000 (oldest deadline 2019-12-08T19:50:56.898793+0000) 2019-12-08T19:50:58.240 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:50:58.235+0000 1694f700 -1 osd.5 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:14.848107+0000 front 2019-12-08T19:50:14.859242+0000 (oldest deadline 2019-12-08T19:50:56.504707+0000) 2019-12-08T19:50:58.498 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:50:58.493+0000 1694f700 -1 osd.7 146 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:50:11.945614+0000 front 2019-12-08T19:50:11.948572+0000 (oldest deadline 2019-12-08T19:50:56.124550+0000) 2019-12-08T19:51:16.132 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:07:47.559 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.136 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:07:47.562 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.136 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:07:47.564 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.155 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:07:47.580 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.160 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:07:47.753 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.162 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:07:47.755 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.249 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:07:47.675 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.260 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:07:47.688 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:16.294 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:07:47.886 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:51:49.047 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:49.044+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:50.054 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:50.045+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:51.085 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:51.082+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:52.099 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:52.095+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:53.088 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:53.084+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:54.095 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:54.091+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:55.122 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:55.119+0000 1694f700 -1 osd.0 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:10.527441+0000 front 2019-12-08T19:51:10.253287+0000 (oldest deadline 2019-12-08T19:51:55.119374+0000) 2019-12-08T19:51:55.124 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:55.121+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:55.765 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:51:55.759+0000 1694f700 -1 osd.2 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:09.405407+0000 front 2019-12-08T19:51:09.653054+0000 (oldest deadline 2019-12-08T19:51:55.096271+0000) 2019-12-08T19:51:56.079 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:56.074+0000 1694f700 -1 osd.0 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:10.527441+0000 front 2019-12-08T19:51:10.253287+0000 (oldest deadline 2019-12-08T19:51:55.119374+0000) 2019-12-08T19:51:56.081 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:56.078+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:56.650 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:51:56.643+0000 1694f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:11.414794+0000 front 2019-12-08T19:51:11.403941+0000 (oldest deadline 2019-12-08T19:51:56.048159+0000) 2019-12-08T19:51:56.784 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:51:56.779+0000 1694f700 -1 osd.2 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:09.405407+0000 front 2019-12-08T19:51:09.653054+0000 (oldest deadline 2019-12-08T19:51:55.096271+0000) 2019-12-08T19:51:57.122 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:57.118+0000 1694f700 -1 osd.0 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:10.527441+0000 front 2019-12-08T19:51:10.253287+0000 (oldest deadline 2019-12-08T19:51:55.119374+0000) 2019-12-08T19:51:57.123 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:57.119+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:57.691 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:51:57.685+0000 1694f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:11.414794+0000 front 2019-12-08T19:51:11.403941+0000 (oldest deadline 2019-12-08T19:51:56.048159+0000) 2019-12-08T19:51:57.799 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:51:57.795+0000 1694f700 -1 osd.2 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:09.405407+0000 front 2019-12-08T19:51:09.653054+0000 (oldest deadline 2019-12-08T19:51:55.096271+0000) 2019-12-08T19:51:57.896 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:51:57.889+0000 1694f700 -1 osd.4 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:11.313283+0000 front 2019-12-08T19:51:11.412540+0000 (oldest deadline 2019-12-08T19:51:57.155479+0000) 2019-12-08T19:51:58.131 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:58.124+0000 1694f700 -1 osd.0 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:10.527441+0000 front 2019-12-08T19:51:10.253287+0000 (oldest deadline 2019-12-08T19:51:55.119374+0000) 2019-12-08T19:51:58.132 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:58.128+0000 1694f700 -1 osd.0 148 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:51:58.307 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:51:58.303+0000 1694f700 -1 osd.3 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:14.312868+0000 front 2019-12-08T19:51:14.218458+0000 (oldest deadline 2019-12-08T19:51:58.293953+0000) 2019-12-08T19:51:58.650 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:51:58.644+0000 1694f700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:14.238338+0000 front 2019-12-08T19:51:14.308827+0000 (oldest deadline 2019-12-08T19:51:58.168169+0000) 2019-12-08T19:51:58.682 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:51:58.677+0000 1694f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:11.414794+0000 front 2019-12-08T19:51:11.403941+0000 (oldest deadline 2019-12-08T19:51:56.048159+0000) 2019-12-08T19:51:58.814 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:51:58.808+0000 1694f700 -1 osd.2 148 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:51:09.405407+0000 front 2019-12-08T19:51:09.653054+0000 (oldest deadline 2019-12-08T19:51:55.096271+0000) 2019-12-08T19:51:59.602 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:51:59.587+0000 1694f700 -1 osd.0 149 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:107373 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422562592 gen 1] snapc 0=[] ondisk+write+known_if_redirected e148) 2019-12-08T19:52:00.806 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T19:52:00.769+0000 20e25700 -1 RGWWatcher::handle_error cookie 5422562592 err (107) Transport endpoint is not connected 2019-12-08T19:53:04.324 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:53:04.278+0000 30f85700 -1 osd.1 157 build_incremental_map_msg missing incremental map 157 2019-12-08T19:53:26.213 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:09:57.641 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.222 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:09:57.650 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.228 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:09:57.654 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.233 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:09:57.826 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.233 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:09:57.659 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.241 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:09:57.832 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.368 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:09:57.794 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.376 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:09:57.804 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:26.382 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:09:57.975 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:53:34.067 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:53:33.803+0000 1694f700 -1 osd.6 158 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e156) 2019-12-08T19:53:35.212 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:53:34.981+0000 1694f700 -1 osd.6 158 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e156) 2019-12-08T19:53:36.198 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:53:36.163+0000 1694f700 -1 osd.6 158 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e156) 2019-12-08T19:54:26.731 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:26.726+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:27.316 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:27.306+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:27.588 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:27.584+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:27.601 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:27.595+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:27.775 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:27.772+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:28.347 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:28.343+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:28.557 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:28.551+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:28.620 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:28.616+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:28.825 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:28.820+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:28.831 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:28.824+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:29.314 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:29.310+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:29.475 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:29.470+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:29.573 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:29.566+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:29.661 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:29.657+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:29.782 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:29.776+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:29.846 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:29.840+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:30.288 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:30.284+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:30.486 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:30.481+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:30.554 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:30.550+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:30.615 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:30.609+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:30.687 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:30.683+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:30.790 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:30.785+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:30.894 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:30.888+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:31.340 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:31.337+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:31.470 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:31.465+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:31.515 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:31.511+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:31.582 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:31.576+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:31.719 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:31.714+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:31.816 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:31.811+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:31.856 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:31.851+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:32.364 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:32.359+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:32.481 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:32.476+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:32.535 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:32.529+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:32.605 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:32.599+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:32.688 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:32.683+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:32.796 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:32.792+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:32.867 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:32.862+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:33.327 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:33.321+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:33.505 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:33.491+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:33.526 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:33.520+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:33.585 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:33.579+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:33.666 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:33.660+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:33.840 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:33.834+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:33.882 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:33.877+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:34.379 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:34.374+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:34.477 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:34.472+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:34.562 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:34.555+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:34.595 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:34.589+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:34.657 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:34.652+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:34.841 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:34.834+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:34.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:34.855+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:35.389 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:35.384+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:35.500 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:35.494+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:35.516 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:35.510+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:35.600 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:35.594+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:35.615 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:35.611+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:35.844 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:35.839+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:35.864 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:35.858+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:36.413 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:36.408+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:36.464 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:36.458+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:36.509 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:36.504+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:36.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:36.549+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:36.585 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:36.579+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:36.813 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:36.808+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:36.884 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:36.878+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:37.397 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:37.391+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:37.420 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:37.414+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:37.544 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:37.539+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:37.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:37.549+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:37.557 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:37.552+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:37.794 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:37.788+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:37.851 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:37.845+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:38.416 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:38.411+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:38.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:38.440+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:38.504 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:38.498+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:38.537 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:38.531+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:38.554 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:38.548+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:38.821 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:38.816+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:38.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:38.872+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:39.389 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:39.385+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:39.471 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:39.465+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:39.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:39.506+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:39.513 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:39.509+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:39.562 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:39.558+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:39.824 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:39.820+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:39.869 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:39.863+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:40.354 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:40.350+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:40.464 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:40.458+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:40.477 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:40.474+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:40.534 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:40.529+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:40.564 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:40.560+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:40.828 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:40.822+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:40.847 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:40.843+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:41.392 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:41.387+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:41.432 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:41.426+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:41.443 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:41.439+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:41.520 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:41.516+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:41.523 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:41.517+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:41.825 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:41.819+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:41.839 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:41.835+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:42.387 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:42.382+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:42.429 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:42.424+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:42.461 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:42.456+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:42.523 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:42.516+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:42.564 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:42.560+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:42.860 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:42.856+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:42.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:42.856+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:43.399 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:43.395+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:43.400 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:43.396+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:43.422 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:43.417+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:43.482 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:43.476+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:43.542 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:43.538+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:43.835 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:43.831+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:43.916 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:43.906+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:44.366 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:44.362+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:44.409 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:44.404+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:44.435 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:44.432+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:44.476 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:44.470+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:44.594 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:44.589+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:44.821 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:44.817+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:44.951 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:44.946+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:45.377 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:45.370+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:45.381 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:45.377+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:45.451 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:45.446+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:45.466 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:45.461+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:45.555 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:45.550+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:45.866 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:45.862+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:45.941 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:45.934+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:54:46.343 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:54:46.337+0000 1694f700 -1 osd.7 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.487212+0000 front 2019-12-08T19:53:45.568056+0000 (oldest deadline 2019-12-08T19:54:29.055604+0000) 2019-12-08T19:54:46.372 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:54:46.368+0000 1694f700 -1 osd.1 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.122703+0000 front 2019-12-08T19:53:46.871140+0000 (oldest deadline 2019-12-08T19:54:26.845097+0000) 2019-12-08T19:54:46.404 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:54:46.399+0000 1694f700 -1 osd.0 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.944712+0000 front 2019-12-08T19:53:45.914029+0000 (oldest deadline 2019-12-08T19:54:29.989679+0000) 2019-12-08T19:54:46.483 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:54:46.477+0000 1694f700 -1 osd.4 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:41.918231+0000 front 2019-12-08T19:53:42.718339+0000 (oldest deadline 2019-12-08T19:54:27.317214+0000) 2019-12-08T19:54:46.572 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:54:46.569+0000 1694f700 -1 osd.2 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:44.924352+0000 front 2019-12-08T19:53:45.589004+0000 (oldest deadline 2019-12-08T19:54:27.022303+0000) 2019-12-08T19:54:46.874 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:54:46.870+0000 1694f700 -1 osd.3 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:47.334755+0000 front 2019-12-08T19:53:46.746577+0000 (oldest deadline 2019-12-08T19:54:26.659105+0000) 2019-12-08T19:54:46.939 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:54:46.934+0000 1694f700 -1 osd.5 158 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:53:45.324005+0000 front 2019-12-08T19:53:42.661133+0000 (oldest deadline 2019-12-08T19:54:28.367480+0000) 2019-12-08T19:55:44.339 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:44.289+0000 1694f700 -1 osd.0 164 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:44.744 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:44.584+0000 1694f700 -1 osd.3 164 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:45.200 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:45.196+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:45.388 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:45.344+0000 1694f700 -1 osd.0 165 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:45.508 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:45.503+0000 1694f700 -1 osd.7 165 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:45.821 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:45.816+0000 1694f700 -1 osd.3 165 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:46.236 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:46.228+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:46.370 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:46.366+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:46.371 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:46.367+0000 1694f700 -1 osd.0 166 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:46.504 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:46.498+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:46.797 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:46.792+0000 1694f700 -1 osd.3 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:47.217 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:47.213+0000 1694f700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:47.261 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:47.256+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:47.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:47.407+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:47.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:47.408+0000 1694f700 -1 osd.0 166 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:47.483 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:47.465+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:47.835 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:47.831+0000 1694f700 -1 osd.3 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:48.140 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:48.136+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:48.222 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:48.218+0000 1694f700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:48.243 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:48.239+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:48.387 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:48.383+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:48.389 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:48.386+0000 1694f700 -1 osd.0 166 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:48.503 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:48.499+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:48.803 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:48.798+0000 1694f700 -1 osd.3 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:49.179 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:49.174+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:49.224 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:49.217+0000 1694f700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:49.285 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:49.280+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:49.357 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:49.352+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:49.363 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:49.359+0000 1694f700 -1 osd.0 166 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:49.557 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:49.531+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:49.819 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:49.813+0000 1694f700 -1 osd.3 166 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:50.192 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:50.188+0000 1694f700 -1 osd.2 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:50.240 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:50.236+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:50.275 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:50.271+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:50.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:50.338+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:50.344 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:50.341+0000 1694f700 -1 osd.0 167 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:50.594 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:50.591+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:50.868 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:50.861+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:51.167 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:51.164+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:51.195 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:51.190+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:51.316 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:51.311+0000 1694f700 -1 osd.5 167 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:51.338 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:51.333+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:51.341 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:51.335+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:51.558 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:51.553+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:51.853 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:51.849+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:52.167 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:52.163+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:52.234 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:52.221+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:52.322 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:52.315+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:52.343 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:52.339+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:52.345 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:52.342+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:52.562 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:52.558+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:52.892 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:52.888+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:53.145 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:53.142+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:53.256 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:53.250+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:53.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:53.340+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:53.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:53.342+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:53.355 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:53.346+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:53.566 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:53.560+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:53.877 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:53.872+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:54.105 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:54.101+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:54.301 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:54.295+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:54.354 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:54.348+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:54.356 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:54.351+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:54.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:54.356+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:54.599 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:54.584+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:54.889 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:54.883+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:55.133 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:55.129+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:55.290 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:55.284+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:55.358 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:55.353+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:55.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:55.356+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:55.380 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:55.374+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:55.631 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:55.617+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:55.875 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:55.870+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:56.136 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:56.130+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:56.300 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:56.294+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:56.366 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:56.361+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:56.367 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:56.364+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:56.377 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:56.371+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:56.612 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:56.605+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:56.870 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:56.865+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:57.189 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:57.185+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:57.297 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:57.290+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:57.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:57.347+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:57.353 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:57.350+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:57.381 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:57.374+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:57.664 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:57.657+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:57.876 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:57.871+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:58.217 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:58.212+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:58.344 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:58.337+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:58.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:58.355+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:58.360 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:58.357+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:58.422 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:58.416+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:58.648 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:58.642+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:58.834 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:58.830+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:55:59.232 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:55:59.228+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:55:59.328 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:59.323+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:55:59.330 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:55:59.327+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:55:59.378 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:55:59.372+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:55:59.381 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:55:59.375+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:55:59.707 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:55:59.701+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:55:59.809 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:55:59.805+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:56:00.280 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:56:00.276+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:56:00.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:00.304+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:56:00.313 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:00.309+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:56:00.382 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:56:00.377+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:56:00.398 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:56:00.392+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:56:00.733 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:56:00.727+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:56:00.778 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:56:00.775+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:56:01.244 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:56:01.240+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:56:01.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:01.319+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:56:01.325 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:01.321+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:56:01.376 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:56:01.370+0000 1694f700 -1 osd.5 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.280644+0000 front 2019-12-08T19:55:02.103210+0000 (oldest deadline 2019-12-08T19:55:44.372788+0000) 2019-12-08T19:56:01.439 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:56:01.433+0000 1694f700 -1 osd.4 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:01.542243+0000 front 2019-12-08T19:55:01.624815+0000 (oldest deadline 2019-12-08T19:55:46.715022+0000) 2019-12-08T19:56:01.750 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:56:01.742+0000 1694f700 -1 osd.7 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.321954+0000 front 2019-12-08T19:55:00.527559+0000 (oldest deadline 2019-12-08T19:55:44.961009+0000) 2019-12-08T19:56:01.790 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:56:01.786+0000 1694f700 -1 osd.3 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:00.058678+0000 front 2019-12-08T19:55:00.057867+0000 (oldest deadline 2019-12-08T19:55:44.051429+0000) 2019-12-08T19:56:02.231 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:56:02.227+0000 1694f700 -1 osd.2 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:03.081673+0000 front 2019-12-08T19:55:03.082707+0000 (oldest deadline 2019-12-08T19:55:47.744018+0000) 2019-12-08T19:56:02.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:02.323+0000 1694f700 -1 osd.0 168 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:55:02.995512+0000 front 2019-12-08T19:55:03.001518+0000 (oldest deadline 2019-12-08T19:55:43.981787+0000) 2019-12-08T19:56:02.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:02.325+0000 1694f700 -1 osd.0 168 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:56:03.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:56:03.332+0000 1694f700 -1 osd.0 169 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:107913 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472307696 gen 1] snapc 0=[] ondisk+write+known_if_redirected e160) 2019-12-08T19:56:04.670 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T19:56:04.579+0000 20e25700 -1 RGWWatcher::handle_error cookie 1472307696 err (107) Transport endpoint is not connected 2019-12-08T19:56:33.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:56:33.201+0000 1694f700 -1 osd.3 170 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:108054 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 821294176 gen 46] snapc 0=[] ondisk+write+known_if_redirected e169) 2019-12-08T19:57:21.985 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:13:53.402 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:21.986 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:13:53.413 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:21.997 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:13:53.590 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:22.006 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:13:53.432 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:22.010 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:13:53.600 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:22.013 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:13:53.439 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:22.258 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:13:53.686 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:22.268 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:13:53.690 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:22.294 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:13:53.887 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:57:55.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:57:55.148+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:57:55.904 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:57:55.897+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:57:56.001 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:57:55.997+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:57:56.132 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:57:56.128+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:57:56.466 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:57:56.463+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:57:56.781 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:57:56.776+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:57:56.943 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:57:56.938+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:57:56.961 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:57:56.957+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:57:57.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:57:57.159+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:57:57.451 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:57:57.447+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:57:57.786 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:57:57.779+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:57:57.977 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:57:57.973+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:57:57.989 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:57:57.985+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:57:57.989 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:57:57.984+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:57:58.076 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:57:58.070+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:57:58.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:57:58.198+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:57:58.470 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:57:58.462+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:57:58.837 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:57:58.832+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:57:59.001 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:57:58.997+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:57:59.026 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:57:59.020+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:57:59.028 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:57:59.025+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:57:59.085 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:57:59.079+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:57:59.174 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:57:59.170+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:57:59.463 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:57:59.457+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:57:59.827 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:57:59.820+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:00.011 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:00.006+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:00.041 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:00.037+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:00.053 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:00.048+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:00.111 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:00.105+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:00.175 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:00.170+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:00.442 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:00.438+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:00.836 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:00.831+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:01.042 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:01.039+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:01.058 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:01.047+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:01.075 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:01.069+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:01.136 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:01.129+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:01.220 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:01.216+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:01.461 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:01.457+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:01.863 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:01.857+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:02.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:02.031+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:02.069 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:02.062+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:02.094 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:02.090+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:02.124 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:02.118+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:02.204 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:02.200+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:02.476 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:02.471+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:02.867 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:02.851+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:03.086 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:03.082+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:03.111 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:03.104+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:03.123 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:03.119+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:03.124 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:03.118+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:03.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:03.225+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:03.472 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:03.468+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:03.909 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:03.904+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:04.115 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:04.110+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:04.116 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:04.112+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:04.127 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:04.120+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:04.154 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:04.147+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:04.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:04.197+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:04.474 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:04.468+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:04.940 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:04.934+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:05.096 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:05.092+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:05.108 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:05.103+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:05.136 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:05.132+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:05.173 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:05.168+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:05.198 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:05.194+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:05.510 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:05.505+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:05.982 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:05.977+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:06.100 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:06.095+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:06.128 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:06.124+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:06.135 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:06.131+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:06.186 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:06.174+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:06.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:06.193+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:06.559 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:06.555+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:06.992 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:06.986+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:07.079 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:07.074+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:07.092 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:07.088+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:07.119 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:07.115+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:07.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:07.147+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:07.213 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:07.208+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:07.539 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:07.534+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:07.981 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:07.975+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:08.057 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:08.050+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:08.083 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:08.079+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:08.117 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:08.113+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:08.128 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:08.124+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:08.200 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:08.192+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:08.546 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:08.541+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:08.954 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:08.948+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:09.101 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:09.096+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:09.109 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:09.104+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:09.122 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:09.118+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:09.142 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:09.138+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:09.175 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:09.169+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:09.554 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:09.550+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:09.969 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:09.963+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:10.091 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:10.088+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:10.119 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:10.113+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:10.124 INFO:teuthology.orchestra.run.smithi129.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-12-08T19:58:10.133 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:10.127+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:10.144 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:10.134+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:10.207 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:10.202+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:10.580 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:10.576+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:10.975 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:10.969+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:11.085 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:11.081+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:11.114 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:11.109+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:11.153 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:11.144+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:11.172 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:11.166+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:11.189 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:11.179+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:11.545 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:11.541+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:12.000 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:11.995+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:12.055 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:12.052+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:12.138 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:12.132+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:12.150 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:12.146+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:12.181 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:12.176+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:12.194 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:12.189+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:12.528 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:12.525+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:12.958 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:12.953+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:13.036 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:13.032+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:13.107 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:13.102+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:13.170 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:13.165+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:13.188 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:13.182+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:13.203 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:13.198+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:13.500 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:13.497+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:13.995 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:13.990+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:13.997 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:13.993+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:14.159 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:14.153+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:14.161 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:14.157+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:14.223 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:14.219+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:14.235 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:14.230+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:14.528 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:14.524+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:14.990 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:14.985+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:15.022 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:15.018+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:15.130 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:15.126+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:15.154 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:15.149+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:15.189 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:15.185+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:15.261 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:15.252+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:15.523 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:15.516+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-12-08T19:58:16.009 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:16.006+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:16.042 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:16.036+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:16.114 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:58:16.108+0000 1694f700 -1 osd.0 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.636357+0000 front 2019-12-08T19:57:13.015932+0000 (oldest deadline 2019-12-08T19:57:54.294018+0000) 2019-12-08T19:58:16.172 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:16.166+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:16.184 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:16.181+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:16.300 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:16.295+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:16.523 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:16.520+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:17.049 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:17.044+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:17.055 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:17.050+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:17.182 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:17.178+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:17.217 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:17.212+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:17.345 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:17.340+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:17.525 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:17.521+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:18.049 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:18.044+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:18.089 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:18.086+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:18.173 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:18.163+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:18.196 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:18.192+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:18.355 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:18.350+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:18.540 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:18.535+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:19.051 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:19.047+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:19.087 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:19.082+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:19.162 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:19.157+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:19.191 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:19.186+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:19.326 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:19.321+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:19.554 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:19.548+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:20.032 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:20.027+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:20.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:20.116+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:20.140 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:20.136+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:20.199 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:20.194+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:20.362 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:20.357+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:20.514 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:20.510+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:21.008 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:21.002+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:21.108 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:21.104+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:21.121 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:21.117+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:21.180 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:21.174+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:21.322 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:21.317+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:21.520 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:21.514+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:58:21.977 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T19:58:21.963+0000 1694f700 -1 osd.1 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466898+0000 front 2019-12-08T19:57:14.612269+0000 (oldest deadline 2019-12-08T19:57:55.348353+0000) 2019-12-08T19:58:22.081 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:58:22.076+0000 1694f700 -1 osd.7 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:14.614557+0000 front 2019-12-08T19:57:13.522069+0000 (oldest deadline 2019-12-08T19:57:56.679212+0000) 2019-12-08T19:58:22.101 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:58:22.096+0000 1694f700 -1 osd.5 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.940033+0000 front 2019-12-08T19:57:14.737934+0000 (oldest deadline 2019-12-08T19:57:55.782781+0000) 2019-12-08T19:58:22.150 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:58:22.145+0000 1694f700 -1 osd.2 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:12.972712+0000 front 2019-12-08T19:57:12.578882+0000 (oldest deadline 2019-12-08T19:57:57.068936+0000) 2019-12-08T19:58:22.333 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:58:22.329+0000 1694f700 -1 osd.4 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:11.769149+0000 front 2019-12-08T19:57:14.737587+0000 (oldest deadline 2019-12-08T19:57:57.524777+0000) 2019-12-08T19:58:22.513 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:58:22.508+0000 1694f700 -1 osd.3 174 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T19:57:13.466779+0000 front 2019-12-08T19:57:12.957983+0000 (oldest deadline 2019-12-08T19:57:55.768965+0000) 2019-12-08T19:59:21.027 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:21.020+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:21.257 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:21.252+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:21.696 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:21.692+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:21.753 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:21.745+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:21.824 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:21.820+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:22.047 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:22.041+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:22.308 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:22.303+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:22.672 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:22.667+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:22.793 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:22.787+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:22.796 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:22.792+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:23.032 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:23.022+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:23.319 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:23.315+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:23.693 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:23.689+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:23.808 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:23.802+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:23.846 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:23.842+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:23.991 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:23.985+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:24.324 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:24.318+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:24.355 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:24.351+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:24.723 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:24.719+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:24.793 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:24.786+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:24.852 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:24.843+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:25.002 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:24.996+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:25.292 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:25.285+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:25.353 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:25.348+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:25.698 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:25.693+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:25.806 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:25.799+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:25.868 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:25.864+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:25.869 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:25.867+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:26.052 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:26.045+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:26.317 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:26.310+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:26.368 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:26.364+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:26.723 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:26.712+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:26.820 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:26.812+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:26.910 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:26.904+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:26.915 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:26.912+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:27.049 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:27.043+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:27.315 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:27.307+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:27.395 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:27.391+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:27.694 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:27.689+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:27.787 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:27.780+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:27.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:27.940+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:27.947 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:27.943+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:28.048 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:28.042+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:28.281 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:28.270+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:28.401 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:28.395+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:28.687 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:28.683+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:28.817 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:28.810+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:28.919 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:28.914+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:28.920 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:28.916+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:29.074 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:29.067+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:29.301 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:29.294+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:29.373 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:29.369+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:29.645 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:29.641+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:29.841 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:29.833+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:29.934 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:29.930+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:29.934 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:29.931+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:30.028 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:30.022+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:30.261 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:30.255+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:30.375 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:30.371+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:30.686 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:30.682+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:30.837 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:30.830+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:30.966 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:30.961+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:30.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:30.963+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:31.003 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:30.995+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:31.302 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:31.295+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:31.413 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:31.409+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:31.706 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:31.701+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:31.802 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:31.795+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:31.964 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:31.957+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:31.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:31.973+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:31.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:31.975+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:32.260 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:32.254+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:32.446 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:32.441+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:32.725 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:32.721+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:32.802 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:32.795+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:32.960 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:32.955+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:32.962 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:32.959+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:32.997 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:32.991+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:33.083 INFO:teuthology.orchestra.run.smithi129.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T19:59:33.306 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:33.299+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:33.474 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:33.470+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:33.734 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:33.731+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:33.777 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:33.770+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:33.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:33.977+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:33.983 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:33.978+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:34.023 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:34.011+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:34.312 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:34.306+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:34.428 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:34.424+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:34.712 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:34.707+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:34.756 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:34.748+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:35.022 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:35.017+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:35.025 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:35.021+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:35.042 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:35.035+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:35.344 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:35.338+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:35.443 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:35.438+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:35.720 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:35.716+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:35.785 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:35.778+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:36.014 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:36.008+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:36.023 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:36.014+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:36.027 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:36.018+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:36.396 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:36.388+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:36.424 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:36.418+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:36.726 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:36.722+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:36.765 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:36.758+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:36.974 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:36.967+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:36.985 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:36.979+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:36.985 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:36.982+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:37.409 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:37.405+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:37.413 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:37.407+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:37.708 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:37.704+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:37.803 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:37.796+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:37.818 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:37.813+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:37.930 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:37.922+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:38.024 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:38.019+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:38.025 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:38.021+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:38.095 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:16:09.523 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.133 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:16:09.561 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.142 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:16:09.567 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.178 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:16:09.603 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.185 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:16:09.778 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.199 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:16:09.792 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.282 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:16:09.709 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.305 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:16:09.730 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.415 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:38.404+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:38.435 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:38.429+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:38.438 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:16:10.030 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T19:59:38.680 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:38.675+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:38.783 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:38.777+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:38.788 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:38.784+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:38.958 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:38.948+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:39.004 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:39.000+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:39.004 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:39.001+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:39.392 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:39.387+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:39.462 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:39.456+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:39.698 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:39.694+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:39.797 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:39.793+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:39.825 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:39.820+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:39.976 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:39.972+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:39.977 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:39.974+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:39.982 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:39.976+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:40.347 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:40.343+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:40.463 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:40.455+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:40.702 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:40.697+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:40.781 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:40.775+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:40.832 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:40.826+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:40.966 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:40.959+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:40.971 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:40.966+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:40.975 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:40.972+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:41.335 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:41.329+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:41.447 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:41.441+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:41.741 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:41.737+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:41.776 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:41.771+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:41.834 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:41.827+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:41.979 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:41.975+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:41.980 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:41.977+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:41.994 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:41.988+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:42.352 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:42.348+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:42.489 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:42.483+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:42.750 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:42.744+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:42.764 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:42.760+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:42.860 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:42.852+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:42.985 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:42.981+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:42.986 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:42.983+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:43.042 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:43.036+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:43.363 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:43.356+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:43.503 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:43.491+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:43.739 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:43.723+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:43.743 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:43.738+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:43.783 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:43.778+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:43.832 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:43.824+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:44.025 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:44.020+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:44.028 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:44.025+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:44.050 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:44.043+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:44.392 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:44.388+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:44.477 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:44.470+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:44.766 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:44.761+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:44.787 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:44.774+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:44.790 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:44.778+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:44.824 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:44.817+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:44.997 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:44.992+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:44.997 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:44.994+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:45.029 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:45.021+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:45.359 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:45.354+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:45.528 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:45.521+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:45.779 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:45.763+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:45.788 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:45.783+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:45.808 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:45.800+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:45.860 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T19:59:45.867 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:45.861+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:46.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:46.033+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:46.040 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:46.036+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:46.074 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:46.068+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:46.357 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:46.353+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:46.487 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:46.481+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:46.770 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:46.763+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:46.776 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:46.768+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:46.831 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:46.825+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:46.843 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:46.839+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:47.064 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:47.057+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:47.070 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:47.064+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:47.071 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:47.067+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:47.377 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:47.373+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:47.498 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:47.491+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:47.789 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:47.783+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:47.811 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:47.803+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:47.812 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:47.807+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:47.821 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:47.817+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:48.071 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:48.067+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:48.071 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:48.068+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:48.112 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:48.103+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:48.413 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:48.409+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:48.485 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:48.477+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:48.783 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:48.778+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:48.824 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:48.818+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:48.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:48.828+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:48.837 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:48.831+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:49.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:49.035+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:49.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:49.037+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:49.081 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:49.075+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:49.440 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:49.434+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:49.460 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:49.455+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:49.795 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:49.789+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:49.826 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:49.819+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:49.859 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:49.851+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:49.860 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:49.854+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:50.042 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:50.034+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:50.044 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:50.035+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:50.067 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:50.058+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:50.421 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:50.416+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:50.432 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:50.423+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:50.783 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:50.776+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:50.802 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:50.796+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:50.828 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:50.821+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:50.831 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:50.824+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:51.029 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:51.022+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:51.075 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:51.067+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:51.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:51.071+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:51.417 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:51.412+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:51.424 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:51.416+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:51.786 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:51.779+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:51.786 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:51.781+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:51.806 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:51.800+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:51.812 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:51.804+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:52.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:52.026+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:52.033 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:52.029+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:52.060 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:52.053+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:52.435 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:52.430+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:52.457 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:52.449+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:52.829 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:52.821+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:52.831 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:52.826+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:52.838 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:52.832+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:52.842 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:52.837+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:52.995 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:52.987+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:52.996 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:52.991+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:53.049 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:53.043+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:53.127 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T19:59:53.109+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e180 v180) 2019-12-08T19:59:53.479 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:53.474+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:53.498 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:53.492+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:53.799 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:53.794+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:53.814 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:53.808+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:53.848 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:53.841+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:53.850 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:53.844+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:53.985 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:53.977+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:53.986 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:53.981+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:54.083 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:54.076+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:54.458 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:54.452+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:54.475 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:54.468+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:54.799 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:54.793+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:54.813 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:54.807+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:54.895 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:54.889+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:54.905 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:54.894+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:54.988 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:54.983+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:54.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:54.985+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:55.069 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:55.062+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:55.454 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:55.448+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:55.488 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:55.482+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:55.753 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:55.748+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:55.818 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:55.812+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:55.853 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:55.846+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:55.854 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:55.848+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:55.962 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:55.956+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:55.972 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:55.962+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:56.078 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:56.073+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:56.448 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:56.441+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:56.497 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:56.492+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:56.754 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:56.749+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:56.838 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:56.832+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:56.849 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:56.837+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:56.859 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:56.852+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:56.943 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:56.938+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:56.944 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:56.941+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:57.120 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:57.110+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:57.467 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:57.463+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:57.499 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:57.493+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:57.739 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:57.733+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:57.844 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:57.836+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:57.846 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:57.840+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:57.862 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:57.856+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:57.984 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:57.979+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:57.985 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:57.982+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:58.078 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:58.073+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:58.128 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T19:59:58.125+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e180 v180) 2019-12-08T19:59:58.518 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:58.512+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:58.520 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:58.515+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:58.704 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:58.700+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:58.835 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:58.828+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:58.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:58.831+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:58.913 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:58.902+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T19:59:59.030 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:59.026+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T19:59:59.031 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T19:59:59.028+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T19:59:59.171 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T19:59:59.166+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T19:59:59.522 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T19:59:59.516+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T19:59:59.558 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T19:59:59.554+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T19:59:59.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T19:59:59.666+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T19:59:59.879 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:59.873+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T19:59:59.881 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T19:59:59.876+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T19:59:59.935 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T19:59:59.929+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:00.052 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:00.047+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:00.052 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:00.049+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:00.223 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:00.217+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:00.482 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:00.475+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:00.610 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:00.606+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:00.696 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:00.691+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:00.930 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:00.924+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:00.931 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:00.927+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:00.960 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:00.952+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:01.099 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:01.094+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:01.101 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:01.096+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:01.196 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:01.190+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:01.486 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:01.481+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:01.633 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:01.618+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:01.663 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:01.658+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:01.891 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:01.885+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:01.892 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:01.887+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:02.004 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:01.998+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:02.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:02.110+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:02.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:02.115+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:02.198 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:02.193+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:02.454 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:02.448+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:02.640 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:02.634+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:02.653 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:02.647+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:02.892 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:02.887+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:02.895 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:02.891+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:02.958 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:02.952+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:03.141 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:00:03.135+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e180 v180) 2019-12-08T20:00:03.145 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:03.140+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:03.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:03.144+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:03.212 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:03.206+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:03.498 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:03.494+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:03.595 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:03.590+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:03.661 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:03.656+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:03.863 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:03.858+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:03.867 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:03.863+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:03.941 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:03.936+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:04.189 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:04.182+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:04.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:04.186+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:04.226 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:04.220+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:04.467 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:04.462+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:04.605 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:04.601+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:04.653 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:04.650+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:04.895 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:04.890+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:04.899 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:04.895+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:04.917 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:04.912+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:05.194 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:05.189+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:05.195 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:05.191+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:05.270 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:05.264+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:05.455 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:05.449+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:05.615 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:05.610+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:05.695 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:05.690+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:05.891 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:05.884+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:05.920 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:05.912+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:05.921 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:05.917+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:06.229 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:06.223+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:06.236 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:06.229+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:06.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:06.232+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:06.447 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:06.442+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:06.591 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:06.587+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:06.706 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:06.700+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:06.877 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:06.870+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:06.878 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:06.873+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:06.916 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:06.909+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:07.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:07.194+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:07.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:07.197+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:07.209 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:07.204+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:07.418 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:07.413+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:07.612 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:07.604+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:07.674 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:07.669+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:07.851 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:07.845+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:07.855 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:07.851+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:07.966 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:07.961+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:08.162 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:00:08.158+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e180 v180) 2019-12-08T20:00:08.178 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:08.173+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:08.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:08.185+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:08.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:08.187+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:08.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:08.380+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:08.564 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:08.559+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:08.700 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:08.695+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:08.846 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:08.840+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:08.847 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:08.843+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:08.946 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:08.941+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:09.171 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:09.167+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:09.172 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:09.168+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:09.194 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:09.190+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:09.374 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:09.369+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:09.599 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:09.590+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:09.668 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:09.663+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:09.849 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:09.844+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:09.853 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:09.849+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:09.934 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:09.928+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:10.146 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:10.140+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:10.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:10.142+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:10.249 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:10.243+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:10.332 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:10.327+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:10.608 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:10.604+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:10.719 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:10.715+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:10.868 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:10.860+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:10.869 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:10.863+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:10.916 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:10.909+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:11.177 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:11.171+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:11.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:11.174+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:11.285 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:11.280+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:11.387 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:11.381+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:11.577 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:11.573+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:11.735 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:11.730+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:11.823 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:11.817+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:11.825 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:11.819+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:11.969 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:11.963+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:12.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:12.165+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:12.170 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:12.166+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:12.265 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:12.258+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:12.350 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:00:12.343+0000 1694f700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.922195+0000 front 2019-12-08T19:58:37.924275+0000 (oldest deadline 2019-12-08T19:59:23.418489+0000) 2019-12-08T20:00:12.624 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:00:12.620+0000 1694f700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.979410+0000 front 2019-12-08T19:58:37.980570+0000 (oldest deadline 2019-12-08T19:59:21.139805+0000) 2019-12-08T20:00:12.739 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:00:12.734+0000 1694f700 -1 osd.2 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.256813+0000 front 2019-12-08T19:58:37.257478+0000 (oldest deadline 2019-12-08T19:59:21.202573+0000) 2019-12-08T20:00:12.874 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:12.868+0000 1694f700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T19:59:03.418068+0000 (oldest deadline 2019-12-08T19:59:43.418068+0000) 2019-12-08T20:00:12.878 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:00:12.873+0000 1694f700 -1 osd.6 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e178) 2019-12-08T20:00:12.949 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:00:12.943+0000 1694f700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:39.703378+0000 front 2019-12-08T19:58:39.676611+0000 (oldest deadline 2019-12-08T19:59:20.739534+0000) 2019-12-08T20:00:13.172 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:13.168+0000 1694f700 -1 osd.0 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:37.709064+0000 front 2019-12-08T19:58:37.658335+0000 (oldest deadline 2019-12-08T19:59:21.433543+0000) 2019-12-08T20:00:13.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:13.170+0000 1694f700 -1 osd.0 180 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:108586 5.5 5:a93a5511:::notify.2:head [watch ping cookie 386331152 gen 1] snapc 0=[] ondisk+write+known_if_redirected e176) 2019-12-08T20:00:13.214 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:00:13.207+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e180 v180) 2019-12-08T20:00:13.228 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:00:13.223+0000 1694f700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T19:58:36.199967+0000 front 2019-12-08T19:58:36.201060+0000 (oldest deadline 2019-12-08T19:59:20.982099+0000) 2019-12-08T20:00:14.158 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:14.153+0000 1694f700 -1 osd.0 181 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108627 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 386331152 gen 2] snapc 0=[] ondisk+write+known_if_redirected e177) 2019-12-08T20:00:15.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:00:15.238+0000 1694f700 -1 osd.0 182 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:108627 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 386331152 gen 2] snapc 0=[] ondisk+write+known_if_redirected e177) 2019-12-08T20:00:16.091 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:00:16.086+0000 20e25700 -1 RGWWatcher::handle_error cookie 386331152 err (107) Transport endpoint is not connected 2019-12-08T20:01:14.564 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:17:45.992 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.573 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:17:46.000 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.584 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:17:46.010 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.595 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:17:46.018 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.600 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:17:46.193 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.607 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:17:46.198 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.706 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:17:46.132 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.726 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:17:46.153 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:14.799 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:17:46.392 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:01:33.602 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:33.597+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:34.645 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:34.640+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:35.431 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:35.428+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:35.547 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:35.539+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:35.609 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:35.603+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:36.409 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:36.403+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:36.540 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:36.535+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:36.570 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:36.564+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:36.646 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:36.636+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:36.926 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:36.921+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:37.377 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:37.373+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:37.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:37.533+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:37.572 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:37.566+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:37.617 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:37.610+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:37.947 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:37.943+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:38.322 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:38.317+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:38.367 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:38.363+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:38.409 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:38.404+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:38.535 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:38.528+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:38.564 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:38.558+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:38.593 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:38.588+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:38.973 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:38.968+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:39.352 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:39.347+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:39.374 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:39.367+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:39.391 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:39.387+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:39.512 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:39.507+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:39.517 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:39.512+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:39.625 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:39.620+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:39.971 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:39.967+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:40.348 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:40.344+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:40.382 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:40.379+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:40.442 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:40.436+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:40.466 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:40.461+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:40.489 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:40.484+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:40.620 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:40.616+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:40.941 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:40.937+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:41.304 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:41.300+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:41.376 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:41.372+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:41.472 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:41.467+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:41.481 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:41.477+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:41.484 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:41.480+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:41.669 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:41.659+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:41.991 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:41.986+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:42.314 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:42.309+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:42.362 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:42.354+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:42.462 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:42.458+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:42.486 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:42.481+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:42.515 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:42.511+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:42.702 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:42.698+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:43.023 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:43.019+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:43.283 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:43.279+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:43.395 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:43.391+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:43.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:43.452+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:43.489 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:43.484+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:43.492 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:43.488+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:43.754 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:43.751+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:44.066 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:44.062+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:44.289 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:44.285+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:44.406 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:44.402+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:44.472 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:44.468+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:44.490 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:44.485+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:44.526 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:44.520+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:44.810 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:44.805+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:45.085 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:45.080+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:45.289 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:45.286+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:45.378 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:45.374+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:45.480 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:45.476+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:45.504 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:45.499+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:45.511 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:45.502+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:45.788 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:45.784+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:46.095 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:46.091+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:46.253 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:46.249+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:46.376 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:46.372+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:46.471 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:46.467+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:46.528 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:46.520+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:46.535 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:46.532+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:46.750 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:46.745+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:47.072 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:47.068+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:47.263 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:47.257+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:47.402 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:47.398+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:47.505 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:47.500+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:47.547 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:47.543+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:47.564 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:47.560+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:47.802 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:47.795+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:48.033 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:48.027+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:48.308 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:48.303+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:48.438 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:48.435+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:48.463 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:48.459+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:48.513 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:48.508+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:48.589 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:48.585+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:48.762 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:48.757+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:49.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:49.067+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:49.335 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:49.331+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:49.439 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:49.436+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:49.493 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:49.488+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:49.552 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:49.544+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:49.555 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:49.550+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:49.728 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:49.723+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:50.105 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:50.101+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:50.384 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:50.380+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:50.475 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:50.471+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:50.494 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:50.489+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:50.523 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:50.519+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:50.564 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:50.559+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:50.753 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:50.748+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:51.059 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:51.054+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:51.431 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:51.427+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:51.454 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:51.450+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:51.528 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:51.523+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:51.532 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:51.517+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:51.603 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:51.599+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:51.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:51.762+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:52.072 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:52.069+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:52.443 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:52.438+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:52.479 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:52.474+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:52.518 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:52.512+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:52.597 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:52.585+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:52.597 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:52.593+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:52.768 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:52.763+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:53.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:53.067+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:53.449 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:53.445+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:53.498 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:53.494+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:53.547 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:53.542+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:53.550 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:53.545+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:53.589 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:53.585+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:53.802 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:53.796+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:54.093 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:54.090+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:54.410 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:54.405+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:54.493 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:54.489+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:54.517 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:54.512+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:54.598 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:54.592+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:54.615 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:54.611+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:54.837 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:54.832+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:55.112 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:55.105+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:55.373 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:55.369+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:55.475 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:55.469+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:55.518 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:55.513+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:55.581 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:55.577+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:55.624 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:55.619+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:55.864 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:55.858+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:56.146 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:56.142+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:56.417 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:56.412+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:56.483 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:56.476+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:56.519 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:56.516+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:56.613 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:56.609+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:56.662 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:56.657+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:56.865 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:56.858+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:57.151 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:57.147+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:57.464 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:57.458+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:57.497 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:57.491+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:57.561 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:57.558+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:57.613 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:57.608+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:57.627 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:57.621+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:57.866 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:57.860+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:57.870 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:57.865+0000 1694f700 -1 osd.7 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:01:58.175 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:58.171+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:58.457 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:58.454+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:58.533 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:58.527+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:58.612 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:58.607+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:58.646 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:58.642+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:58.699 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:58.676+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:58.885 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:58.880+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:58.887 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:58.882+0000 1694f700 -1 osd.7 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:01:59.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:01:59.202+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:01:59.452 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:01:59.448+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:01:59.516 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:01:59.511+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:01:59.651 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:01:59.647+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:01:59.653 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:01:59.647+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:01:59.688 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:01:59.682+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:01:59.938 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:59.932+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:01:59.938 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:01:59.932+0000 1694f700 -1 osd.7 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:02:00.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:02:00.208+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:02:00.431 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:00.427+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:02:00.542 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:02:00.536+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:02:00.614 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:02:00.610+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:02:00.637 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:02:00.633+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:02:00.721 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:02:00.715+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:02:00.895 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:00.889+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:02:00.896 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:00.890+0000 1694f700 -1 osd.7 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:02:01.216 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:02:01.211+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:02:01.402 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:01.397+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:02:01.515 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:02:01.510+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:02:01.595 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:02:01.591+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:02:01.651 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:02:01.647+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:02:01.725 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:02:01.719+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:02:01.897 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:01.890+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:02:01.897 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:01.892+0000 1694f700 -1 osd.7 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:02:02.193 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:02:02.189+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:02:02.428 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:02.424+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:02:02.509 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:02:02.503+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:02:02.649 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:02:02.644+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:02:02.697 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:02:02.693+0000 1694f700 -1 osd.2 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.109651+0000 front 2019-12-08T20:00:53.742874+0000 (oldest deadline 2019-12-08T20:01:37.989432+0000) 2019-12-08T20:02:02.732 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:02:02.726+0000 1694f700 -1 osd.4 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.411246+0000 front 2019-12-08T20:00:51.769336+0000 (oldest deadline 2019-12-08T20:01:35.731361+0000) 2019-12-08T20:02:02.922 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:02.916+0000 1694f700 -1 osd.7 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.766919+0000 front 2019-12-08T20:00:49.194165+0000 (oldest deadline 2019-12-08T20:01:33.593832+0000) 2019-12-08T20:02:02.924 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:02.920+0000 1694f700 -1 osd.7 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:02:03.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:02:03.190+0000 1694f700 -1 osd.3 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:52.632600+0000 front 2019-12-08T20:00:52.553288+0000 (oldest deadline 2019-12-08T20:01:35.999658+0000) 2019-12-08T20:02:03.448 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:03.445+0000 1694f700 -1 osd.1 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.597882+0000 front 2019-12-08T20:00:51.691765+0000 (oldest deadline 2019-12-08T20:01:34.889760+0000) 2019-12-08T20:02:03.491 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:02:03.485+0000 1694f700 -1 osd.5 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:51.939861+0000 front 2019-12-08T20:00:51.772517+0000 (oldest deadline 2019-12-08T20:01:35.369554+0000) 2019-12-08T20:02:03.637 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:02:03.633+0000 1694f700 -1 osd.0 185 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:00:53.492678+0000 front 2019-12-08T20:00:52.100224+0000 (oldest deadline 2019-12-08T20:01:37.984656+0000) 2019-12-08T20:02:04.355 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:04.271+0000 1694f700 -1 osd.7 186 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:02:05.387 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:02:05.302+0000 1694f700 -1 osd.7 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109044 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29394 in=29394b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e185) 2019-12-08T20:02:42.923 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:02:42.867+0000 11caf700 -1 osd.2 189 build_incremental_map_msg missing incremental map 189 2019-12-08T20:02:43.043 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:02:43.016+0000 33f8b700 -1 osd.2 189 build_incremental_map_msg missing incremental map 189 2019-12-08T20:02:43.121 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:02:43.098+0000 33f8b700 -1 osd.2 189 build_incremental_map_msg missing incremental map 189 2019-12-08T20:02:45.237 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:45.215+0000 32788700 -1 osd.1 190 build_incremental_map_msg missing incremental map 190 2019-12-08T20:02:45.543 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:45.477+0000 36790700 -1 osd.1 190 build_incremental_map_msg missing incremental map 190 2019-12-08T20:02:45.642 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:45.613+0000 35f8f700 -1 osd.1 190 build_incremental_map_msg missing incremental map 190 2019-12-08T20:02:46.782 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:46.590+0000 33f8b700 -1 osd.1 191 build_incremental_map_msg missing incremental map 190 2019-12-08T20:02:48.958 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:19:20.386 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:48.976 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:19:20.402 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:48.977 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:19:20.405 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:48.979 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:19:20.405 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:49.077 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:19:20.669 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:49.085 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:19:20.677 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:49.086 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:19:20.513 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:49.145 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:19:20.570 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:49.283 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:19:20.875 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:02:50.398 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:50.342+0000 37f93700 -1 osd.1 191 build_incremental_map_msg missing incremental map 157 2019-12-08T20:02:50.774 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:50.611+0000 37f93700 -1 osd.1 191 build_incremental_map_msg missing incremental map 173 2019-12-08T20:02:51.675 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:02:51.578+0000 37f93700 -1 osd.1 191 build_incremental_map_msg missing incremental map 190 2019-12-08T20:03:01.347 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:01.341+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:01.774 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:01.768+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:01.959 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:01.956+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:02.370 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:02.364+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:02.770 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:02.763+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:02.920 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:02.917+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:03.337 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:03.331+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:03.454 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:03.448+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:03.807 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:03.801+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:03.910 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:03.907+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:04.335 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:04.328+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:04.429 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:04.422+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:04.789 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:04.784+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:04.848 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:04.841+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:04.942 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:04.939+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:05.332 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:05.325+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:05.464 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:05.457+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:05.808 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:05.803+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:05.901 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:05.897+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:05.903 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:05.897+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:05.982 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:05.976+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:05.982 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:05.979+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:06.331 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:06.324+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:06.436 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:06.429+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:06.794 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:06.789+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:06.915 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:06.911+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:06.928 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:06.921+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:06.942 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:06.937+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:06.943 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:06.939+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:07.329 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:07.323+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:07.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:07.442+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:07.772 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:07.767+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:07.885 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:07.878+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:07.938 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:07.931+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:07.979 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:07.974+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:07.980 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:07.976+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:08.376 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:08.370+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:08.498 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:08.491+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:08.804 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:08.800+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:08.906 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:08.897+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:08.944 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:08.938+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:08.945 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:08.941+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:08.968 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:08.963+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:09.381 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:09.375+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:09.548 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:09.541+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:09.812 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:09.809+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:09.942 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:09.935+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:09.989 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:09.985+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:09.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:09.984+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:09.993 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:09.990+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:10.375 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:10.368+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:10.527 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:10.521+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:10.782 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:10.773+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:10.953 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:10.945+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:10.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:10.968+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:10.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:10.971+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:11.004 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:10.999+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:11.353 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:11.346+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:11.532 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:11.526+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:11.769 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:11.762+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:11.982 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:11.978+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:11.985 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:11.977+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:12.011 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:12.006+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:12.013 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:12.010+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:12.344 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:12.337+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:12.562 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:12.552+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:12.741 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:12.737+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:12.977 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:12.971+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:12.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:12.974+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:12.989 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:12.982+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:13.010 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:13.005+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:13.382 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:13.375+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:13.590 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:13.583+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:13.749 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:13.741+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:13.783 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:13.778+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:13.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:13.978+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:13.982 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:13.979+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:13.987 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:13.980+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:14.040 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:14.037+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:14.422 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:14.414+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:14.633 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:14.626+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:14.752 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:14.747+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:14.790 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:14.785+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:14.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:14.962+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:14.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:14.965+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:14.978 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:14.972+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:15.054 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:15.049+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:15.381 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:15.374+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:15.606 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:15.599+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:15.790 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:15.785+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:15.806 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:15.800+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:15.994 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:15.991+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:15.996 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:15.993+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:15.997 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:15.991+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:16.098 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:16.095+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:16.350 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:16.343+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:16.627 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:16.620+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:16.835 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:16.830+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:16.848 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:16.843+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:16.992 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:16.985+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:17.008 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:17.003+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:17.009 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:17.005+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:17.085 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:17.080+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:17.310 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:17.303+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:17.637 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:17.632+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:17.807 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:17.801+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:17.810 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:17.806+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:17.996 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:17.989+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:17.996 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:17.993+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:18.030 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:18.024+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:18.130 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:18.125+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:18.272 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:18.265+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:18.613 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:18.607+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:18.806 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:18.801+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:18.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:18.831+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:18.993 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:18.989+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:18.994 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:18.992+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:19.023 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:19.015+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:19.087 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:19.084+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:19.312 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:19.305+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:19.629 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:19.612+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:19.845 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:19.839+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:19.849 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:19.842+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:19.989 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:19.983+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:19.996 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:19.990+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:19.997 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:19.993+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:20.079 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:20.074+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:20.316 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:20.309+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:20.637 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:20.629+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:20.817 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:20.811+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:20.831 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:20.824+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:20.832 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:20.826+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:21.038 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:21.032+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:21.040 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:21.033+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:21.051 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:21.044+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:21.052 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:21.048+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:21.301 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:21.295+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:21.649 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:21.643+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:21.790 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:21.785+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:21.876 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:21.869+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:21.877 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:21.872+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:22.017 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:22.012+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:22.049 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:22.043+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:22.092 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:22.088+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:22.094 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:22.091+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:22.354 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:22.347+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:22.676 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:22.671+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:22.773 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:22.770+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:22.878 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:22.872+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:22.879 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:22.875+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:23.027 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:23.018+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:23.033 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:23.025+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:23.115 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:23.109+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:23.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:23.112+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:23.319 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:23.312+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:23.652 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:23.646+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:23.804 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:23.799+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:23.874 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:23.867+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:23.875 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:23.871+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:24.036 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:24.031+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:24.071 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:24.067+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:24.112 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:24.108+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:24.113 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:24.110+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:24.287 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:24.282+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:24.695 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:24.689+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:24.835 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:24.830+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:24.878 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:24.873+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:24.882 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:24.878+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:25.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:25.031+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:25.045 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:25.039+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:25.133 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:25.127+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:25.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:25.130+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:25.324 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:25.319+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:25.717 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:25.711+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:25.848 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:25.843+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:25.914 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:25.907+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:25.915 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:25.910+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:26.000 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:25.995+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:26.010 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:25.995+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:26.176 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:26.171+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:26.177 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:26.173+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:26.288 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:26.282+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:26.724 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:26.718+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:26.889 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:26.877+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:26.935 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:26.928+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:26.938 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:26.931+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:26.992 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:26.987+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:27.000 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:26.994+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:27.221 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:27.216+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:27.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:27.220+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:27.251 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:27.245+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:27.755 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:27.750+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:27.906 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:27.902+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:27.961 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:27.954+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:27.962 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:27.957+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:27.986 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:27.981+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:27.994 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:27.988+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:28.216 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:28.210+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:28.218 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:28.213+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:28.267 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:28.261+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:28.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:28.787+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:28.899 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:28.894+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:28.971 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:28.965+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:28.987 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:28.982+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:29.012 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:29.006+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:29.015 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:29.012+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:29.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:29.263+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:29.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:29.265+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:29.285 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:29.280+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:29.773 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:29.767+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:29.870 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:29.865+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:29.993 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:29.989+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:30.001 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:29.994+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:30.023 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:30.018+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:30.027 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:30.023+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:30.304 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:30.298+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:30.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:30.301+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:30.337 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:30.331+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:30.739 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:30.733+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:30.826 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:30.823+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:30.957 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:30.951+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:31.026 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:31.020+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:31.028 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:31.023+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:31.029 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:31.025+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:31.325 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:31.319+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:31.327 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:31.322+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:31.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:31.324+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:31.754 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:31.748+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:31.813 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:31.808+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:31.971 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:31.957+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:32.000 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:31.996+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:32.013 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:32.007+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:32.015 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:32.009+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:32.282 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:32.276+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:32.283 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:32.279+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:32.377 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:32.368+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:32.757 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:32.750+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:32.830 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:32.826+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:32.979 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:32.974+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:32.984 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:32.979+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:32.985 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:32.981+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:33.001 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:32.997+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:33.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:33.289+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:33.295 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:33.292+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:33.416 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:33.410+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:33.778 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:33.772+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:33.867 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:33.863+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:33.992 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:33.984+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:33.995 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:33.990+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:34.005 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:33.998+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:34.027 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:34.023+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:34.061 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:03:34.058+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 40sec e191 v191) 2019-12-08T20:03:34.275 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:34.270+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:34.280 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:34.276+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:34.404 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:34.392+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:34.823 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:34.811+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:34.833 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:34.828+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:35.023 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:35.018+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:35.043 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:35.035+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:35.044 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:35.038+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:35.049 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:35.041+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:35.278 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:35.273+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:35.281 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:35.277+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:35.401 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:35.394+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:35.870 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:35.863+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:35.887 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:35.880+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:35.999 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:35.992+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:36.041 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:36.034+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:36.046 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:36.038+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:36.047 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:36.041+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:36.286 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:36.282+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:36.288 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:36.284+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:36.452 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:36.445+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:36.828 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:36.821+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:36.926 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:36.920+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:36.959 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:36.952+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:37.077 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:37.070+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:37.093 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:37.086+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:37.096 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:37.091+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:37.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:37.277+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:37.286 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:37.280+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:37.413 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:37.404+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:37.876 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:37.869+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:37.967 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:37.962+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:37.972 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:37.967+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:38.034 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:38.028+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:38.051 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:38.044+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:38.052 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:38.046+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:38.338 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:38.330+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:38.339 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:38.334+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:38.411 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:38.404+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:38.890 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:38.883+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:38.966 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:38.962+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:38.984 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:38.978+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:39.057 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:39.050+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:39.084 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:39.076+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:39.085 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:39.079+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:39.086 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:03:39.079+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 40sec e191 v191) 2019-12-08T20:03:39.383 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:39.378+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:39.387 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:39.382+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:39.396 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:39.390+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:39.887 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:39.880+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:39.942 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:39.936+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:39.955 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:39.949+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:40.048 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:40.040+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:40.049 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:40.043+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:40.057 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:40.051+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:40.379 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:40.372+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:40.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:40.393+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:40.401 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:40.396+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:40.872 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:40.865+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:40.946 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:40.939+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:40.962 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:40.956+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:41.085 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:41.077+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:41.086 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:41.080+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:41.111 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:41.104+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:41.341 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:41.334+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:41.381 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:41.374+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:41.382 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:41.377+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:41.905 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:41.897+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:41.995 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:41.989+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:41.998 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:41.991+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:42.079 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:42.072+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:42.080 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:42.075+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:42.133 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:42.128+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:42.304 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:42.297+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:42.402 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:42.398+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:42.405 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:42.401+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:42.921 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:42.914+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:42.989 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:42.984+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:43.037 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:43.033+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:43.065 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:43.057+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:43.066 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:43.060+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:43.134 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:43.127+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:43.262 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:43.254+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:43.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:43.405+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:43.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:43.406+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:43.891 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:43.884+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:44.002 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:43.996+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:44.028 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:44.020+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:44.029 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:44.023+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:44.045 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:44.038+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:44.093 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:44.086+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:44.107 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:03:44.100+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 40sec e191 v191) 2019-12-08T20:03:44.271 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:44.264+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:44.419 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:44.413+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:44.421 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:44.417+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:44.855 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:44.848+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:44.998 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:44.990+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:45.000 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:44.993+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:45.013 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:45.008+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:45.030 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:45.025+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:45.089 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:45.078+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:45.299 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:45.292+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:45.427 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:45.422+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:45.428 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:45.424+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:45.844 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:45.837+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:45.993 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:45.988+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:46.040 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:46.034+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:46.043 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:46.039+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:46.047 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:46.043+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:46.085 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:46.079+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:46.339 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:46.333+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:46.433 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:46.427+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:46.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:46.430+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:46.839 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:46.832+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:46.972 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:46.968+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:47.073 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:47.065+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:47.073 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:47.068+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:47.074 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:47.068+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:47.134 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:47.125+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:47.360 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:47.352+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:47.449 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:47.441+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:47.451 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:47.446+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:47.810 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:47.804+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:47.994 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:47.990+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:48.034 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:48.029+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:48.044 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:48.036+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:48.044 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:48.039+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:48.183 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:48.174+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:48.404 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:03:48.397+0000 1694f700 -1 osd.4 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.443790+0000 front 2019-12-08T20:02:20.430315+0000 (oldest deadline 2019-12-08T20:03:01.321696+0000) 2019-12-08T20:03:48.417 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:48.412+0000 1694f700 -1 osd.0 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.153699+0000 front 2019-12-08T20:02:20.118731+0000 (oldest deadline 2019-12-08T20:03:05.789438+0000) 2019-12-08T20:03:48.417 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:48.414+0000 1694f700 -1 osd.0 191 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:48.777 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:03:48.772+0000 1694f700 -1 osd.7 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:19.087831+0000 front 2019-12-08T20:02:19.076902+0000 (oldest deadline 2019-12-08T20:03:03.135813+0000) 2019-12-08T20:03:49.022 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:03:49.017+0000 1694f700 -1 osd.3 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.119058+0000 front 2019-12-08T20:02:20.107957+0000 (oldest deadline 2019-12-08T20:03:04.417569+0000) 2019-12-08T20:03:49.056 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:49.047+0000 1694f700 -1 osd.6 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:02:40.813952+0000 (oldest deadline 2019-12-08T20:03:20.813952+0000) 2019-12-08T20:03:49.058 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:03:49.052+0000 1694f700 -1 osd.6 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:106527 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e189) 2019-12-08T20:03:49.078 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:03:49.075+0000 1694f700 -1 osd.2 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.207996+0000 front 2019-12-08T20:02:20.204318+0000 (oldest deadline 2019-12-08T20:03:04.872446+0000) 2019-12-08T20:03:49.159 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:03:49.149+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 40sec e191 v191) 2019-12-08T20:03:49.232 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:03:49.225+0000 1694f700 -1 osd.5 191 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:02:20.926423+0000 front 2019-12-08T20:02:20.859965+0000 (oldest deadline 2019-12-08T20:03:01.300560+0000) 2019-12-08T20:03:49.463 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:49.401+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:50.378 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:03:50.374+0000 1694f700 -1 osd.0 192 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109188 5.5 5:a93a5511:::notify.2:head [watch ping cookie 901058416 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-12-08T20:03:51.479 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:03:51.453+0000 20e25700 -1 RGWWatcher::handle_error cookie 901058416 err (107) Transport endpoint is not connected 2019-12-08T20:04:19.242 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:04:19.239+0000 3378a700 -1 osd.1 196 build_incremental_map_msg missing incremental map 196 2019-12-08T20:05:14.096 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:14.092+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:14.166 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:14.162+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:14.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:14.252+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:14.422 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:14.417+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:14.515 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:21:45.943 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.526 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:21:45.954 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.533 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:21:45.959 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.539 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:21:45.965 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.544 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:21:46.137 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.556 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:21:46.149 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.746 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:21:46.339 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.768 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:21:46.193 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.774 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:21:46.201 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:05:14.785 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:14.780+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:15.109 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:15.106+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:15.215 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:15.210+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:15.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:15.223+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:15.427 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:15.424+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:15.435 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:15.430+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:15.788 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:15.783+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:16.031 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:16.028+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:16.118 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:16.113+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:16.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:16.176+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:16.232 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:16.225+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:16.396 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:16.392+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:16.415 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:16.409+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:16.792 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:16.787+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:17.062 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:17.057+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:17.124 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:17.119+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:17.183 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:17.178+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:17.193 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:17.188+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:17.389 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:17.385+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:17.434 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:17.428+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:17.752 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:17.746+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:18.095 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:18.090+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:18.108 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:18.102+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:18.170 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:18.164+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:18.195 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:18.192+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:18.409 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:18.403+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:18.430 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:18.425+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:18.708 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:18.702+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:19.091 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:19.087+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:19.152 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:19.149+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:19.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:19.192+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:19.212 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:19.207+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:19.387 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:19.383+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:19.412 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:19.404+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:19.749 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:19.743+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:20.135 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:20.132+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:20.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:20.150+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:20.191 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:20.185+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:20.194 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:20.189+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:20.395 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:20.391+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:20.408 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:20.403+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:20.704 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:20.698+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:21.183 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:21.179+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:21.199 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:21.195+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:21.216 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:21.211+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:21.236 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:21.232+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:21.377 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:21.372+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:21.419 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:21.413+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:21.695 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:21.689+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:22.195 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:22.190+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:22.237 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:22.232+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:22.254 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:22.248+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:22.264 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:22.260+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:22.399 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:22.395+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:22.409 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:22.403+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:22.681 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:22.675+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:23.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:23.199+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:23.217 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:23.211+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:23.229 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:23.225+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:23.230 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:23.225+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:23.368 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:23.360+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:23.437 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:23.434+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:23.728 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:23.723+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:24.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:24.197+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:24.205 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:24.199+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:24.216 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:24.212+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:24.252 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:24.244+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:24.347 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:24.341+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:24.412 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:24.408+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:24.769 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:24.763+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:25.241 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:25.236+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:25.241 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:25.236+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:25.265 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:25.260+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:25.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:25.265+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:25.312 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:25.306+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:25.390 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:25.386+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:25.798 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:25.793+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:26.230 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:26.226+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:26.240 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:26.236+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:26.241 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:26.237+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:26.266 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:26.261+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:26.291 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:26.283+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:26.405 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:26.402+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:26.784 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:26.778+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:27.211 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:27.207+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:27.244 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:27.230+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:27.253 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:27.250+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:27.282 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:27.279+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:27.323 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:27.317+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:27.432 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:27.428+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:27.792 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:27.787+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:28.202 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:28.196+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:28.239 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:28.235+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:28.241 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:28.236+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:28.257 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:28.253+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:28.302 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:28.296+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:28.410 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:28.405+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:28.769 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:28.763+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:29.170 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:29.163+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:29.224 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:29.215+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:29.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:29.256+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:29.270 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:29.266+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:29.329 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:29.324+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:29.424 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:29.419+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:29.809 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:29.803+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:30.160 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:30.153+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:30.263 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:30.257+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:30.263 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:30.258+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:30.283 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:30.270+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:30.363 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:30.358+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:30.426 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:30.422+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:30.841 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:30.835+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:31.163 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:31.158+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:31.220 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:31.216+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:31.251 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:31.247+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:31.278 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:31.274+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:31.352 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:31.347+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:31.463 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:31.459+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:31.840 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:31.835+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:32.195 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:32.191+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:32.219 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:32.212+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:32.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:32.256+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:32.280 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:32.276+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:32.398 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:32.392+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:32.479 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:32.475+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:32.871 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:32.864+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:33.204 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:33.200+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:33.219 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:33.215+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:33.252 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:33.246+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:33.269 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:33.263+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:33.421 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:33.414+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:33.533 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:33.529+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:33.836 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:33.829+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:34.185 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:34.182+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:34.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:34.263+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:34.270 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:34.264+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:34.283 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:34.279+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:34.424 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:34.418+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:34.491 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:34.487+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:34.885 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:34.879+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:35.212 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:35.208+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:35.257 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:35.252+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:35.274 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:35.270+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:35.305 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:35.301+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:35.423 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:35.418+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:35.529 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:35.525+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:35.846 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:35.838+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:36.216 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:36.213+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:36.249 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:36.246+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:36.297 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:36.292+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:36.361 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:36.355+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:36.481 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:36.467+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:36.548 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:36.544+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:36.888 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:36.882+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:37.211 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:37.206+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:37.249 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:37.246+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:37.342 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:37.330+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:37.398 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:37.393+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:37.486 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:37.481+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:37.505 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:37.500+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:37.926 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:37.920+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:38.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:38.205+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:38.292 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:38.288+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:38.353 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:38.349+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:38.416 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:38.411+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:38.490 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:38.484+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:38.549 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:38.544+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:38.948 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:38.942+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:39.185 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:39.181+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:39.296 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:39.292+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:39.356 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:39.350+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:39.430 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:39.427+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:39.467 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:39.461+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:39.599 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:39.596+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:39.969 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:39.963+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:40.233 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:40.229+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:40.283 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:40.279+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:40.352 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:40.347+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:40.405 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:40.402+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:40.502 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:40.495+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:40.610 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:40.607+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:40.930 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:40.924+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:41.246 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:41.241+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:41.316 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:41.311+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:41.372 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:41.366+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:41.402 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:41.398+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:41.540 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:41.534+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:41.653 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:41.648+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:41.959 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:41.953+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:42.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:42.197+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:42.320 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:42.317+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:42.376 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:42.372+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:42.412 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:42.406+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:42.586 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:42.581+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:42.642 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:42.637+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:42.973 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:42.969+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:43.217 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:43.211+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:43.279 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:43.275+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:43.349 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:43.346+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:43.472 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:43.466+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:43.558 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:43.551+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:43.610 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:43.606+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:44.061 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:43.960+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:44.179 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:44.176+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:44.302 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:44.298+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:44.318 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:44.314+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:44.444 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:44.438+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:44.560 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:44.555+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:44.596 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:44.590+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:44.924 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:44.916+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:45.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:45.225+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:45.285 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:45.280+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:45.296 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:45.291+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:45.491 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:45.486+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:45.563 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:45.557+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:45.633 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:45.627+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:45.911 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:45.906+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:46.234 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:46.230+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:46.251 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:46.246+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:46.275 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:46.270+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:46.501 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:46.495+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:46.552 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:46.545+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:46.660 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:46.657+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:46.897 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:46.891+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:47.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:47.193+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:47.224 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:47.219+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:47.323 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:47.319+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:47.472 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:47.464+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:47.531 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:47.526+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:47.642 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:47.637+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:47.900 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:47.894+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:48.158 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:48.154+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:48.207 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:48.203+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:48.355 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:48.350+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:48.469 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:48.463+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:48.511 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:48.505+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:48.644 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:05:48.639+0000 1694f700 -1 osd.3 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.002449+0000 front 2019-12-08T20:04:29.537668+0000 (oldest deadline 2019-12-08T20:05:15.424257+0000) 2019-12-08T20:05:48.897 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:05:48.891+0000 1694f700 -1 osd.4 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:32.073618+0000 front 2019-12-08T20:04:31.715613+0000 (oldest deadline 2019-12-08T20:05:14.629638+0000) 2019-12-08T20:05:49.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:05:49.157+0000 1694f700 -1 osd.0 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:31.728531+0000 front 2019-12-08T20:04:31.701761+0000 (oldest deadline 2019-12-08T20:05:13.648534+0000) 2019-12-08T20:05:49.175 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:05:49.171+0000 1694f700 -1 osd.2 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.160332+0000 front 2019-12-08T20:04:35.644140+0000 (oldest deadline 2019-12-08T20:05:15.918013+0000) 2019-12-08T20:05:49.372 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:05:49.367+0000 1694f700 -1 osd.1 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:35.186088+0000 front 2019-12-08T20:04:32.409721+0000 (oldest deadline 2019-12-08T20:05:13.564753+0000) 2019-12-08T20:05:49.476 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:05:49.470+0000 1694f700 -1 osd.7 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:30.003665+0000 front 2019-12-08T20:04:30.021704+0000 (oldest deadline 2019-12-08T20:05:13.462945+0000) 2019-12-08T20:05:49.478 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:05:49.473+0000 1694f700 -1 osd.5 196 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:04:28.360145+0000 front 2019-12-08T20:04:28.413459+0000 (oldest deadline 2019-12-08T20:05:13.631874+0000) 2019-12-08T20:05:49.551 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:05:49.544+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 46sec e196 v196) 2019-12-08T20:06:41.669 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:41.662+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:42.680 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:42.676+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:43.675 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:43.671+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:44.674 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:44.670+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:45.646 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:45.642+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:46.688 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:46.682+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:47.153 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:47.148+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:47.419 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:47.413+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:47.631 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:47.627+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:47.731 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:47.727+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:48.130 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:48.127+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:48.402 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:48.396+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:48.599 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:48.591+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:48.779 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:48.775+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:49.124 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:49.120+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:49.451 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:49.445+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:49.564 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:49.559+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:49.571 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:49.566+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:49.760 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:49.756+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:49.762 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:49.760+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:50.129 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:50.125+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:50.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:50.440+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:50.555 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:50.551+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:50.567 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:50.561+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:50.662 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:50.658+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:50.814 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:50.809+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:50.817 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:50.813+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:51.098 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:51.093+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:51.451 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:51.444+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:51.517 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:51.512+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:51.606 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:51.600+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:51.661 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:51.655+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:51.848 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:51.842+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:51.848 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:51.845+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:52.143 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:52.139+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:52.490 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:52.485+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:52.492 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:52.487+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:52.630 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:52.624+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:52.632 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:52.626+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:52.806 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:52.800+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:52.807 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:52.803+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:53.132 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:53.127+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:53.535 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:53.529+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:53.538 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:53.533+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:53.637 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:53.632+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:53.680 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:53.669+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:53.774 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:53.768+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:53.775 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:53.771+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:54.164 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:54.158+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:54.524 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:54.519+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:54.577 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:54.573+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:54.658 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:54.651+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:54.705 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:54.698+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:54.740 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:54.736+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:54.742 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:54.739+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:55.157 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:55.152+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:55.561 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:55.543+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:55.575 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:55.571+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:55.670 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:55.664+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:55.687 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:55.683+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:55.785 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:55.781+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:55.786 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:55.783+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:56.151 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:56.145+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:56.540 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:56.535+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:56.561 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:56.554+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:56.653 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:56.647+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:56.675 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:56.670+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:56.762 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:56.758+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:56.762 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:56.759+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:57.116 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:57.113+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:57.557 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:57.553+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:57.571 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:57.564+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:57.641 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:57.634+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:57.646 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:57.640+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:57.805 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:57.801+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:57.812 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:57.807+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:58.138 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:58.134+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:58.579 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:58.575+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:58.618 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:58.611+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:58.625 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:58.618+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:58.644 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:58.636+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:58.851 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:58.847+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:58.853 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:58.849+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:06:59.186 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:06:59.181+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:06:59.210 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:06:59.204+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:06:59.575 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:06:59.569+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:06:59.599 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:06:59.592+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:06:59.617 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:06:59.611+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:06:59.637 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:06:59.630+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:06:59.831 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:59.827+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:06:59.832 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:06:59.828+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:00.173 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:00.169+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:00.218 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:00.212+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:00.542 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:00.535+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:00.566 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:00.553+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:00.592 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:00.585+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:00.608 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:00.601+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:00.830 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:00.825+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:00.830 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:00.827+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:01.163 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:01.156+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:01.204 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:01.198+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:01.576 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:01.569+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:01.584 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:01.579+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:01.590 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:01.582+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:01.594 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:01.587+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:01.838 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:01.833+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:01.838 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:01.835+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:02.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:02.149+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:02.233 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:02.227+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:02.579 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:02.575+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:02.606 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:02.598+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:02.607 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:02.600+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:02.622 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:02.615+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:02.906 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:02.884+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:02.907 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:02.890+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:03.139 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:03.134+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:03.243 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:03.237+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:03.560 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:03.555+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:03.602 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:03.596+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:03.642 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:03.636+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:03.650 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:03.645+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:03.878 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:03.873+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:03.879 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:03.875+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:04.164 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:04.160+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:04.231 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:04.225+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:04.562 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:04.557+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:04.602 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:04.595+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:04.634 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:04.627+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:04.679 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:04.673+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:04.886 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:04.882+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:04.886 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:04.883+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:05.158 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:05.154+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:05.202 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:05.193+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:05.577 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:05.569+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:05.647 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:05.636+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:05.649 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:05.641+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:05.674 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:05.667+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:05.909 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:05.904+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:05.910 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:05.906+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:06.164 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:06.159+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:06.265 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:06.246+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:06.270 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:06.252+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:06.581 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:06.576+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:06.629 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:06.621+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:06.634 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:06.627+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:06.701 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:06.695+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:06.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:06.941+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:06.947 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:06.943+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:07.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:07.211+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:07.308 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:07.300+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:07.309 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:07.303+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:07.630 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:07.625+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:07.671 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:07.664+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:07.677 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:07.669+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:07.680 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:07.673+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:07.969 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:07.963+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:07.970 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:07.966+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:08.202 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:08.199+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:08.333 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:08.323+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:08.335 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:08.329+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:08.638 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:08.629+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:08.664 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:08.657+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:08.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:08.667+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:08.698 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:08.690+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:09.006 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:09.002+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:09.010 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:09.006+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:09.220 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:09.215+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:09.338 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:09.329+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:09.339 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:09.333+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:09.622 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:09.615+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:09.684 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:09.679+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:09.708 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:09.701+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:09.720 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:09.713+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:10.050 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:10.045+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:10.050 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:10.047+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:10.273 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:10.268+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:10.307 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:10.302+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:10.310 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:10.305+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:10.662 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:10.655+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:10.680 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:10.675+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:10.702 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:10.695+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:10.761 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:10.754+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:11.086 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:11.081+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:11.087 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:11.083+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:11.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:11.226+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:11.295 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:11.287+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:11.297 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:11.290+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:11.622 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:11.612+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:11.636 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:11.632+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:11.672 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:11.665+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:11.794 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:11.788+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:12.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:12.071+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:12.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:12.073+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:12.279 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:12.273+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:12.280 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:12.275+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:12.282 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:12.278+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:12.585 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:12.578+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:12.635 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:12.628+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:12.663 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:12.656+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:12.830 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:12.824+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:13.048 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:13.044+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:13.049 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:13.045+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:13.247 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:13.243+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:13.304 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:13.297+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:13.309 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:13.303+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:13.592 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:13.587+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:13.629 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:13.621+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:13.659 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:13.652+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:13.822 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:13.814+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:14.038 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:14.032+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:14.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:14.037+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:14.249 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:14.245+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:14.288 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:14.280+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:14.289 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:14.283+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:14.586 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:14.578+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:14.594 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:14.589+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:14.635 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:14.628+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:14.821 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:14.813+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:15.087 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:15.082+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:15.091 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:15.087+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:15.220 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:15.215+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:15.274 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:15.268+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:15.277 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:15.272+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:15.564 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:15.558+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:15.585 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:15.580+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:15.643 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:15.636+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:15.776 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:15.768+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:16.069 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:16.064+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:16.072 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:16.068+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:16.269 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:16.265+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:16.277 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:16.271+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:16.281 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:16.276+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:16.568 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:16.562+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:16.579 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:16.575+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:16.655 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:16.648+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:16.767 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:16.758+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:17.110 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:17.104+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:17.111 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:17.107+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:17.241 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:17.233+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:17.242 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:17.236+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:17.250 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:17.245+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:17.576 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:17.569+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:17.581 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:17.578+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:17.703 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:17.696+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:17.767 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:17.760+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:18.072 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:18.068+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:18.073 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:18.069+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:18.247 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:18.241+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:18.250 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:18.246+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:18.274 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:18.271+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:18.568 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:18.563+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:18.593 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:18.586+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:18.749 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:18.743+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:18.761 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:18.756+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:19.057 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:19.051+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:19.057 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:19.054+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:19.218 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:19.209+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:19.219 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:19.211+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:19.325 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:19.321+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:19.551 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:19.547+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:19.599 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:19.591+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:19.765 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:19.758+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:19.782 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:19.774+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:19.800 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:07:19.795+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e202 v202) 2019-12-08T20:07:20.018 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:20.012+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:20.019 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:20.015+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:20.267 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:20.259+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:20.280 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:20.262+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:20.365 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:20.361+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:20.524 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:20.519+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:20.607 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:20.600+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:20.777 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:20.765+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:20.797 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:20.791+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:20.980 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:20.975+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:20.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:20.977+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:21.288 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:21.281+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:21.292 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:21.286+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:21.386 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:21.382+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:21.539 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:21.535+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:21.564 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:21.557+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:21.775 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:21.767+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:21.805 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:21.798+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:21.935 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:21.931+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:21.936 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:21.932+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:22.292 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:22.284+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:22.293 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:22.287+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:22.412 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:22.407+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:22.538 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:22.531+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:22.575 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:22.570+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:22.797 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:22.791+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:22.823 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:22.816+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:22.953 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:22.948+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:22.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:22.953+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:23.295 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:23.287+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:23.296 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:23.290+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:23.384 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:23.379+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:23.565 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:23.559+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:23.605 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:23.598+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:23.805 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:23.797+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:23.838 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:23.832+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:23.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:23.941+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:23.949 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:23.944+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:24.342 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:24.335+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:24.343 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:24.337+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:24.381 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:24.376+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:24.532 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:24.525+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:24.623 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:24.616+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:24.769 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:24.762+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:24.815 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:07:24.811+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e202 v202) 2019-12-08T20:07:24.864 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:24.857+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:24.982 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:24.975+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:24.983 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:24.978+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:25.360 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:25.352+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:25.361 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:25.355+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:25.373 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:25.368+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:25.586 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:25.578+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:25.589 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:25.584+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:25.751 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:25.744+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:25.866 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:25.859+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:25.992 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:25.987+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:25.995 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:25.991+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:26.377 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:26.372+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:26.390 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:26.382+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:26.392 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:26.385+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:26.550 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:26.545+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:26.593 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:26.587+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:26.722 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:26.715+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:26.920 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:26.913+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:26.975 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:26.966+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:26.976 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:26.971+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:27.362 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:27.357+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:27.411 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:27.403+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:27.412 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:27.406+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:27.575 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:27.568+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:27.599 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:27.594+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:27.742 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:27.735+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:27.912 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:27.905+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:27.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:27.951+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:27.961 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:27.956+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:28.395 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:28.387+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:28.396 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:28.390+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:28.396 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:28.391+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:28.565 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:28.559+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:28.582 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:28.576+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:28.717 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:28.710+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:28.935 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:28.928+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:28.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:28.982+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:28.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:28.985+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:29.401 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:29.393+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:29.441 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:29.433+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:29.442 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:29.436+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:29.558 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:29.550+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:29.566 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:29.562+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:29.713 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:29.706+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:29.836 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:07:29.827+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e202 v202) 2019-12-08T20:07:29.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:29.968+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:29.975 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:29.971+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:29.976 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:29.968+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:30.411 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:30.405+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:30.465 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:30.458+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:30.469 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:30.464+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:30.581 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:30.574+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:30.585 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:30.579+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:30.732 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:30.726+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:30.969 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:30.962+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:30.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:30.986+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:30.992 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:30.989+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:31.449 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:31.444+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:31.480 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:31.472+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:31.481 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:31.475+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:31.554 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:31.550+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:31.578 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:31.571+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:31.722 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:31.715+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:31.965 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:31.958+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:32.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:32.034+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:32.040 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:32.036+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:32.448 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:32.443+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:32.480 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:32.472+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:32.481 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:32.475+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:32.512 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:32.508+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:32.628 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:32.623+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:32.725 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:32.718+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:32.958 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:32.951+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:33.024 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:33.020+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:33.030 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:33.025+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:33.469 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:33.465+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:33.502 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:33.495+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:33.505 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:33.500+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:33.514 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:33.509+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:33.650 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:33.643+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:33.748 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:33.741+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:33.951 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:33.944+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:34.060 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:34.054+0000 1694f700 -1 osd.0 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.314891+0000 front 2019-12-08T20:06:03.551099+0000 (oldest deadline 2019-12-08T20:06:49.202225+0000) 2019-12-08T20:07:34.061 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:34.057+0000 1694f700 -1 osd.0 202 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:34.466 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:07:34.462+0000 1694f700 -1 osd.3 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:04.821849+0000 front 2019-12-08T20:06:04.832066+0000 (oldest deadline 2019-12-08T20:06:47.038076+0000) 2019-12-08T20:07:34.478 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:07:34.474+0000 1694f700 -1 osd.2 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.372512+0000 front 2019-12-08T20:06:06.397595+0000 (oldest deadline 2019-12-08T20:06:48.621464+0000) 2019-12-08T20:07:34.555 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:34.548+0000 1694f700 -1 osd.6 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:06:25.258279+0000 (oldest deadline 2019-12-08T20:07:05.258279+0000) 2019-12-08T20:07:34.556 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:07:34.551+0000 1694f700 -1 osd.6 202 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e200) 2019-12-08T20:07:34.636 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:07:34.629+0000 1694f700 -1 osd.7 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:05.711371+0000 front 2019-12-08T20:06:05.710162+0000 (oldest deadline 2019-12-08T20:06:46.756047+0000) 2019-12-08T20:07:34.776 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:07:34.770+0000 1694f700 -1 osd.5 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:03.265535+0000 front 2019-12-08T20:06:03.003639+0000 (oldest deadline 2019-12-08T20:06:47.021961+0000) 2019-12-08T20:07:34.885 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:07:34.871+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e202 v202) 2019-12-08T20:07:34.920 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:07:34.913+0000 1694f700 -1 osd.4 202 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:06:06.217703+0000 front 2019-12-08T20:06:06.333237+0000 (oldest deadline 2019-12-08T20:06:49.692942+0000) 2019-12-08T20:07:35.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:35.100+0000 1694f700 -1 osd.0 203 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:36.067 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:07:36.061+0000 1694f700 -1 osd.0 203 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:109657 5.5 5:a93a5511:::notify.2:head [watch ping cookie 385273936 gen 1] snapc 0=[] ondisk+write+known_if_redirected e198) 2019-12-08T20:07:37.162 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:07:37.144+0000 20e25700 -1 RGWWatcher::handle_error cookie 385273936 err (107) Transport endpoint is not connected 2019-12-08T20:08:05.080 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:24:36.508 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.107 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:24:36.533 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.120 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:24:36.546 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.140 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:24:36.732 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.145 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:24:36.571 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.146 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:24:36.739 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.293 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:24:36.720 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.353 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:24:36.778 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:08:05.389 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:24:36.982 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:01.461 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:01.455+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:01.582 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:01.575+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:01.582 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:01.577+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:01.680 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:01.676+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:02.492 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:02.469+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:02.566 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:02.560+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:02.640 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:02.635+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:02.692 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:02.688+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:03.031 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:03.027+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:03.397 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:03.387+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:03.481 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:03.476+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:03.555 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:03.549+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:03.609 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:03.605+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:03.718 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:03.713+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:04.083 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:04.073+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:04.391 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:04.385+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:04.472 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:04.469+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:04.529 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:04.521+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:04.594 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:04.590+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:04.709 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:04.706+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:05.059 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:05.054+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:05.331 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:05.326+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:05.399 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:05.393+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:05.458 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:05.454+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:05.547 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:05.543+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:05.576 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:05.569+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:05.758 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:05.754+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:06.045 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:06.041+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:06.372 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:06.365+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:06.372 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:06.365+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:06.460 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:06.457+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:06.578 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:06.574+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:06.588 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:06.582+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:06.781 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:06.777+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:07.091 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:07.086+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:07.343 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:07.338+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:07.419 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:07.413+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:07.483 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:07.479+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:07.550 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:07.544+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:07.601 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:07.597+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:07.803 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:07.800+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:08.121 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:08.115+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:08.324 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:08.317+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:08.395 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:08.384+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:08.516 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:08.512+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:08.531 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:08.520+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:08.613 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:08.609+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:08.789 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:08.785+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:09.075 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:09.071+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:09.280 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:09.275+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:09.336 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:25:40.763 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.345 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:25:40.773 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.353 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:25:40.945 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.356 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:25:40.782 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.358 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:25:40.951 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.359 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:25:40.785 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.382 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:09.374+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:09.426 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:25:40.853 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.431 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:25:40.857 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.475 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:25:41.068 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:09:09.481 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:09.478+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:09.553 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:09.548+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:09.636 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:09.633+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:09.763 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:09.760+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:10.046 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:10.042+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:10.301 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:10.296+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:10.344 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:10.339+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:10.484 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:10.479+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:10.575 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:10.570+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:10.672 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:10.668+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:10.750 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:10.746+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:11.072 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:11.068+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:11.263 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:11.256+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:11.349 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:11.333+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:11.518 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:11.515+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:11.574 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:11.558+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:11.692 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:11.687+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:11.743 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:11.739+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:12.091 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:12.087+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:12.286 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:12.281+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:12.371 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:12.366+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:12.491 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:12.487+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:12.565 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:12.559+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:12.719 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:12.715+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:12.743 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:12.735+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:13.118 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:13.106+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:13.320 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:13.315+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:13.395 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:13.390+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:13.478 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:13.472+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:13.543 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:13.535+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:13.687 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:13.683+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:13.760 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:13.756+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:14.150 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:14.146+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:14.357 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:14.350+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:14.358 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:14.353+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:14.524 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:14.517+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:14.552 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:14.546+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:14.684 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:14.680+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:14.773 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:14.769+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:15.129 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:15.124+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:15.375 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:15.370+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:15.396 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:15.390+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:15.483 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:15.480+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:15.585 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:15.579+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:15.638 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:15.634+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:15.792 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:15.789+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:16.118 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:16.095+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:16.342 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:16.335+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:16.425 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:16.419+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:16.465 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:16.461+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:16.624 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:16.618+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:16.646 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:16.641+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:16.815 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:16.810+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:17.120 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:17.116+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:17.328 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:17.322+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:17.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:17.451+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:17.483 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:17.479+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:17.642 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:17.638+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:17.656 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:17.652+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:17.833 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:17.829+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:18.159 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:18.155+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:18.363 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:18.358+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:18.491 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:18.486+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:18.531 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:18.527+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:18.649 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:18.645+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:18.704 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:18.699+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:18.869 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:18.865+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:19.138 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:19.134+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:19.335 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:19.330+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:19.495 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:19.490+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:19.527 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:19.523+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:19.692 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:19.688+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:19.698 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:19.693+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:19.901 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:19.897+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:20.126 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:20.122+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:20.341 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:20.336+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:20.514 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:20.509+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:20.528 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:20.524+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:20.668 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:20.659+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:20.678 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:20.674+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:20.947 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:20.943+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:21.168 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:21.163+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:21.358 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:21.354+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:21.487 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:21.483+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:21.563 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:21.559+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:21.713 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:21.708+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:21.715 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:21.712+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:21.988 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:21.984+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:22.136 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:22.133+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:22.343 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:22.339+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:22.506 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:22.501+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:22.540 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:22.535+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:22.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:22.702+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:22.757 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:22.753+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:23.015 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:23.012+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:23.149 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:23.144+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:23.312 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:23.308+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:23.535 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:23.530+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:23.556 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:23.551+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:23.743 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:23.739+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:23.800 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:23.794+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:24.036 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:24.032+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:24.166 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:24.162+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:24.268 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:24.262+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:24.544 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:24.540+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:24.579 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:24.574+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:24.768 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:24.764+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:24.851 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:24.846+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:25.084 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:25.081+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:25.141 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:25.138+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:25.249 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:25.244+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:25.538 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:25.535+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:25.580 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:25.576+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:25.772 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:25.767+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:25.829 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:25.824+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:26.055 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:26.051+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:26.107 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:26.102+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:26.229 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:26.225+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:26.514 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:26.510+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:26.558 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:26.554+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:26.737 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:26.728+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:26.835 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:26.823+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:27.083 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:27.079+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:27.100 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:27.096+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:27.193 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:27.185+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:27.512 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:27.508+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:27.603 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:27.599+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:27.788 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:27.783+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:27.853 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:27.847+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:28.058 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:28.052+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:28.077 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:28.074+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:28.160 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:28.156+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:28.561 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:28.556+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:28.602 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:28.597+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:28.809 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:28.805+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:28.890 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:28.887+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:29.066 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:29.062+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:29.118 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:29.114+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:29.121 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:29.117+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:29.576 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:29.571+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:29.612 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:29.607+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:29.809 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:29.805+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:29.918 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:29.913+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:30.060 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:30.056+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:30.089 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:30.084+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:30.125 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:30.120+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:30.557 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:30.552+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:30.650 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:30.645+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:30.861 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:30.854+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:30.963 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:30.954+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:31.065 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:31.060+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:31.097 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:31.093+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:31.142 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:31.138+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:31.551 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:31.545+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:31.679 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:31.675+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:31.853 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:31.849+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:31.962 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:31.956+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:32.027 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:32.021+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:32.090 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:32.086+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:32.102 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:32.098+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:32.604 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:32.598+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:32.665 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:32.660+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:32.827 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:32.823+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:32.942 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:32.936+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:33.042 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:33.037+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:33.070 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:33.067+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:33.132 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:33.127+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:33.647 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:33.641+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:33.694 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:33.689+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:33.848 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:33.844+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:33.951 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:33.944+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:34.041 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:34.036+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:34.106 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:34.102+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:34.109 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:34.105+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:34.617 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:34.611+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:09:34.692 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:09:34.688+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:16.399962+0000 front 2019-12-08T20:08:20.864770+0000 (oldest deadline 2019-12-08T20:09:00.836805+0000) 2019-12-08T20:09:34.803 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:09:34.798+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:17.347373+0000 front 2019-12-08T20:08:16.991641+0000 (oldest deadline 2019-12-08T20:09:01.623826+0000) 2019-12-08T20:09:35.001 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:09:34.995+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.409567+0000 front 2019-12-08T20:08:19.412338+0000 (oldest deadline 2019-12-08T20:09:00.790636+0000) 2019-12-08T20:09:35.034 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:09:35.028+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.325009+0000 front 2019-12-08T20:08:20.324159+0000 (oldest deadline 2019-12-08T20:09:02.771764+0000) 2019-12-08T20:09:35.075 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:09:35.072+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.323901+0000 front 2019-12-08T20:08:20.160580+0000 (oldest deadline 2019-12-08T20:09:02.769942+0000) 2019-12-08T20:09:35.124 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:09:35.120+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:20.693018+0000 front 2019-12-08T20:08:15.471986+0000 (oldest deadline 2019-12-08T20:09:00.655411+0000) 2019-12-08T20:09:35.467 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:09:35.459+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 47sec e207 v207) 2019-12-08T20:09:35.595 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:09:35.590+0000 1694f700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:08:19.796806+0000 front 2019-12-08T20:08:19.886464+0000 (oldest deadline 2019-12-08T20:09:04.453946+0000) 2019-12-08T20:10:17.813 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:17.796+0000 124b0700 -1 osd.6 212 build_incremental_map_msg missing incremental map 212 2019-12-08T20:10:17.827 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:17.815+0000 30f85700 -1 osd.6 212 build_incremental_map_msg missing incremental map 212 2019-12-08T20:10:17.989 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:17.946+0000 30f85700 -1 osd.6 212 build_incremental_map_msg missing incremental map 212 2019-12-08T20:10:18.029 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:17.977+0000 30f85700 -1 osd.6 212 build_incremental_map_msg missing incremental map 212 2019-12-08T20:10:18.202 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:18.086+0000 3578e700 -1 osd.6 212 build_incremental_map_msg missing incremental map 212 2019-12-08T20:10:29.700 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:29.695+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:30.726 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:30.721+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:31.747 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:31.743+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:32.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:32.702+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:33.741 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:33.738+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:34.434 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:34.430+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:34.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:34.722+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:34.770 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:34.763+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:34.973 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:34.967+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:35.191 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:35.184+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:35.417 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:35.413+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:35.708 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:35.703+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:35.709 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:35.706+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:35.825 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:35.817+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:35.987 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:35.982+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:36.245 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:36.239+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:36.353 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:36.346+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:36.471 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:36.466+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:36.684 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:36.677+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:36.684 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:36.678+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:36.840 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:36.833+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:36.976 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:36.971+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:37.211 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:37.203+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:37.331 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:37.323+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:37.507 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:37.502+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:37.688 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:37.682+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:37.689 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:37.685+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:37.812 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:37.806+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:37.991 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:37.986+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:38.251 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:38.244+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:38.331 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:38.318+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:38.512 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:38.507+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:38.737 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:38.733+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:38.738 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:38.735+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:38.814 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:38.808+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:38.982 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:38.977+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:39.270 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:39.262+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:39.317 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:39.309+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:39.501 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:39.497+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:39.720 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:39.715+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:39.721 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:39.717+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:39.836 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:39.828+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:39.998 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:39.994+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:40.265 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:40.258+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:40.360 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:40.353+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:40.540 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:40.535+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:40.736 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:40.727+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:40.736 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:40.730+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:40.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:40.784+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:40.979 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:40.975+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:41.245 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:41.238+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:41.414 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:41.407+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:41.555 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:41.550+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:41.782 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:41.777+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:41.783 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:41.780+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:41.829 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:41.822+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:41.988 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:41.985+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:42.257 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:42.250+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:42.436 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:42.429+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:42.595 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:42.591+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:42.818 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:42.815+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:42.819 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:42.817+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:42.881 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:42.874+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:43.033 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:43.028+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:43.220 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:43.212+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:43.414 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:43.407+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:43.559 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:43.555+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:43.858 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:43.853+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:43.859 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:43.855+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:43.879 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:43.870+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:44.007 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:44.001+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:44.242 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:44.226+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:44.394 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:44.387+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:44.532 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:44.527+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:44.841 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:44.829+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:44.885 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:44.881+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:44.886 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:44.882+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:45.039 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:45.035+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:45.219 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:45.211+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:45.352 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:45.346+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:45.509 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:45.504+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:45.794 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:45.788+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:45.935 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:45.930+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:45.943 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:45.932+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:46.085 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:46.081+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:46.268 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:46.261+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:46.395 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:46.388+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:46.461 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:46.458+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:46.615 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:46.609+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:46.767 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:46.760+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:46.897 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:46.892+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:46.898 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:46.894+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:47.119 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:47.115+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:47.278 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:47.271+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:47.402 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:47.395+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:47.451 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:47.439+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:47.583 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:47.578+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:47.782 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:47.775+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:47.935 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:47.931+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:47.936 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:47.933+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:48.170 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:48.166+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:48.254 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:48.247+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:48.382 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:48.375+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:48.481 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:48.477+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:48.548 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:48.542+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:48.820 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:48.812+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:48.976 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:48.970+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:48.977 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:48.973+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:49.182 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:49.177+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:49.260 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:49.243+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:49.392 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:49.385+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:49.442 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:49.437+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:49.535 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:49.529+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:49.861 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:49.855+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:49.938 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:49.933+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:49.938 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:49.935+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:50.160 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:50.156+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:50.253 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:50.246+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:50.401 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:50.394+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:50.430 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:50.425+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:50.501 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:50.495+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:50.892 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:50.887+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:50.970 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:50.966+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:50.973 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:50.970+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:51.208 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:51.201+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:51.209 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:51.205+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:51.404 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:51.399+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:51.416 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:51.403+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:51.473 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:51.467+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:51.917 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:51.909+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:51.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:51.974+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:51.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:51.975+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:52.181 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:52.176+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:52.229 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:52.222+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:52.404 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:52.398+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:52.420 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:52.416+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:52.521 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:52.516+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:52.893 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:52.886+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:52.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:52.983+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:52.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:52.987+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:53.188 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:53.183+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:53.272 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:53.265+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:53.431 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:53.424+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:53.443 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:53.439+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:53.534 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:53.528+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:53.944 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:53.937+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:54.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:54.027+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:54.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:54.033+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:54.231 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:54.225+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:54.237 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:54.230+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:54.401 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:54.398+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:54.465 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:54.458+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:54.544 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:54.538+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:10:54.548 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:54.543+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:54.921 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:54.914+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:55.002 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:54.998+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:55.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:55.000+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:55.224 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:55.219+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:55.283 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:55.276+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:55.377 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:55.373+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:55.462 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:55.455+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:55.542 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:55.533+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:10:55.543 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:55.538+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:55.906 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:55.899+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:56.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:56.025+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:56.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:56.029+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:56.216 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:56.211+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:56.303 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:56.296+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:56.393 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:56.384+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:56.444 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:56.437+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:56.522 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:56.514+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:10:56.523 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:56.517+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:56.863 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:56.855+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:57.080 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:57.076+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:57.084 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:57.080+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:57.230 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:57.226+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:57.337 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:57.331+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:57.422 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:57.418+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:57.494 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:57.487+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:57.516 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:57.509+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:10:57.521 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:57.515+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:57.857 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:57.850+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:58.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:58.034+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:58.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:58.038+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:58.224 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:58.220+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:58.325 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:58.318+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:58.402 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:58.398+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:58.451 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:58.445+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:58.533 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:58.523+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:10:58.534 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:58.528+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:58.868 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:58.861+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:59.017 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:59.012+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:59.017 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:59.014+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:10:59.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:10:59.201+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:10:59.380 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:10:59.372+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:10:59.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:10:59.403+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:10:59.479 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:10:59.472+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:10:59.555 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:59.549+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:10:59.560 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:10:59.554+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:10:59.899 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:10:59.889+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:10:59.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:59.970+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:10:59.977 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:10:59.974+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:00.169 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:00.165+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:00.347 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:00.338+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:00.393 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:00.390+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:00.527 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:00.521+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:00.543 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:00.538+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:00.546 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:00.541+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:00.907 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:00.900+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:00.956 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:00.951+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:00.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:00.952+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:01.213 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:01.208+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:01.382 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:01.376+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:01.411 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:01.405+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:01.499 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:01.494+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:01.518 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:01.511+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:01.519 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:01.514+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:01.917 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:01.913+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:01.924 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:01.921+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:01.935 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:01.921+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:02.247 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:02.243+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:02.368 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:02.362+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:02.380 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:02.377+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:02.467 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:02.462+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:02.547 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:02.540+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:02.551 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:02.546+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:02.960 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:02.956+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:02.961 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:02.958+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:02.978 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:02.970+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:03.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:03.227+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:03.349 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:03.343+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:03.424 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:03.420+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:03.448 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:03.442+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:03.553 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:03.546+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:03.554 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:03.549+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:03.990 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:03.984+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:04.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:04.006+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:04.014 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:04.009+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:04.194 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:04.191+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:04.328 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:04.322+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:04.418 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:04.413+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:04.443 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:04.438+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:04.597 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:04.590+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:04.598 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:04.593+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:05.004 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:04.999+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:05.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:05.001+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:05.032 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:05.026+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:05.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:05.154+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:05.321 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:05.315+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:05.372 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:05.368+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:05.409 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:05.403+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:05.586 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:05.579+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:05.587 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:05.582+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:05.694 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:11:05.688+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e213 v213) 2019-12-08T20:11:05.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:05.986+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:05.994 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:05.990+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:06.073 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:06.068+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:06.166 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:06.162+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:06.353 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:06.347+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:06.377 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:06.358+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:06.426 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:06.419+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:06.582 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:06.575+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:06.583 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:06.578+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:06.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:06.953+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:06.960 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:06.957+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:07.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:07.123+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:07.176 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:07.173+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:07.405 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:07.398+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:07.405 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:07.399+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:07.422 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:07.416+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:07.545 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:07.538+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:07.546 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:07.541+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:07.914 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:07.910+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:07.917 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:07.914+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:08.163 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:08.157+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:08.181 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:08.177+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:08.377 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:08.372+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:08.387 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:08.376+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:08.445 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:08.440+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:08.578 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:08.571+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:08.613 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:08.574+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:08.916 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:08.909+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:08.918 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:08.915+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:09.179 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:09.175+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:09.214 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:09.208+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:09.358 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:09.351+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:09.365 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:09.360+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:09.435 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:09.429+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:09.614 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:09.607+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:09.615 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:09.610+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:09.874 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:09.870+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:09.878 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:09.875+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:10.163 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:10.158+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:10.267 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:10.260+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:10.358 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:10.352+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:10.404 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:10.393+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:10.463 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:10.456+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:10.610 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:10.604+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:10.625 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:10.609+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:10.709 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:11:10.704+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e213 v213) 2019-12-08T20:11:10.901 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:10.892+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:10.902 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:10.897+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:11.157 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:11.151+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:11.319 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:11.313+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:11.382 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:11.376+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:11.406 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:11.401+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:11.436 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:11.431+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:11.656 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:11.649+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:11.657 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:11.653+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:11.892 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:11.883+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:11.892 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:11.888+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:12.180 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:12.174+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:12.341 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:12.335+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:12.404 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:12.399+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:12.416 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:12.410+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:12.452 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:12.446+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:12.649 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:12.642+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:12.650 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:12.645+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:12.936 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:12.929+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:12.936 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:12.932+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:13.172 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:13.166+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:13.300 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:13.294+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:13.394 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:13.387+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:13.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:13.390+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:13.419 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:13.413+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:13.665 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:13.657+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:13.666 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:13.660+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:13.952 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:13.946+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:13.952 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:13.948+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:14.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:14.135+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:14.329 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:14.323+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:14.357 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:14.352+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:14.365 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:14.358+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:14.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:14.419+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:14.685 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:14.679+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:14.688 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:14.683+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:15.002 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:14.994+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:15.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:14.998+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:15.110 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:15.102+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:15.314 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:15.307+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:15.375 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:15.371+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:15.381 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:15.373+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:15.448 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:15.442+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:15.716 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:11:15.712+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e213 v213) 2019-12-08T20:11:15.734 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:15.726+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:15.735 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:15.729+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:15.986 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:15.978+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:15.987 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:15.982+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:16.116 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:16.111+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:16.342 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:16.337+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:16.367 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:16.361+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:16.384 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:16.377+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:16.406 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:16.399+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:16.784 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:16.777+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:16.785 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:16.779+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:17.008 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:17.002+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:17.013 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:17.007+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:17.153 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:17.146+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:17.356 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:17.351+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:17.362 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:17.355+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:17.370 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:17.364+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:17.400 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:17.393+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:17.795 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:17.786+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:17.796 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:17.790+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:17.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:17.984+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:17.991 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:17.987+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:18.156 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:18.150+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:18.331 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:18.325+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:18.347 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:18.344+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:18.381 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:18.373+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:18.393 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:18.386+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:18.764 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:18.757+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:18.765 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:18.760+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:18.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:18.960+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:18.969 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:18.965+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:19.135 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:19.130+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:19.351 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:19.344+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:19.352 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:19.344+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:19.365 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:19.358+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:19.399 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:19.393+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:19.761 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:19.750+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:19.763 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:19.757+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:19.923 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:19.919+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:19.937 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:19.923+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:20.126 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:20.121+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:20.377 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:20.367+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:20.386 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:20.380+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:20.394 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:20.387+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:20.405 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:20.394+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:20.748 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:11:20.743+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e213 v213) 2019-12-08T20:11:20.761 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:20.754+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:20.765 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:20.760+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:20.965 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:20.959+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:20.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:20.962+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:21.111 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:21.107+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:21.364 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:21.353+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:21.381 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:21.368+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:21.382 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:21.370+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:21.400 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:21.396+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:21.835 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:21.828+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:21.843 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:21.836+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:22.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:22.009+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:22.015 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:22.012+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:22.138 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:22.134+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:22.334 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:22.326+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:22.355 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:22.348+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:22.405 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:22.397+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:22.444 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:22.440+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:22.881 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:22.874+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:22.883 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:22.877+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:23.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:22.998+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:23.006 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:23.002+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:23.135 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:23.130+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:23.352 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:23.346+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:23.378 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:23.371+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:23.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:23.393+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:23.411 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:23.397+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:23.905 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:23.884+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:23.905 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:23.887+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:24.043 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:24.028+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:24.044 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:24.031+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:24.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:24.137+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:24.340 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:24.332+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:24.378 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:24.371+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:24.396 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:24.392+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:24.456 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:24.448+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:24.892 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:24.884+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:24.893 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:24.887+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:25.072 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:25.067+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.072471+0000 front 2019-12-08T20:09:51.893106+0000 (oldest deadline 2019-12-08T20:10:34.765846+0000) 2019-12-08T20:11:25.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:25.070+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:25.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:11:25.155+0000 1694f700 -1 osd.3 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:51.349155+0000 front 2019-12-08T20:09:51.271630+0000 (oldest deadline 2019-12-08T20:10:34.886667+0000) 2019-12-08T20:11:25.325 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:11:25.318+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:52.505697+0000 front 2019-12-08T20:09:52.561825+0000 (oldest deadline 2019-12-08T20:10:34.148743+0000) 2019-12-08T20:11:25.335 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:11:25.328+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.401332+0000 front 2019-12-08T20:09:50.524843+0000 (oldest deadline 2019-12-08T20:10:35.075464+0000) 2019-12-08T20:11:25.428 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:11:25.424+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:53.059007+0000 front 2019-12-08T20:09:53.057591+0000 (oldest deadline 2019-12-08T20:10:34.077841+0000) 2019-12-08T20:11:25.429 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:11:25.422+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:09:50.795305+0000 front 2019-12-08T20:09:50.301420+0000 (oldest deadline 2019-12-08T20:10:35.521359+0000) 2019-12-08T20:11:25.821 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:11:25.815+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e213 v213) 2019-12-08T20:11:25.874 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:25.866+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:10:13.904054+0000 (oldest deadline 2019-12-08T20:10:53.904054+0000) 2019-12-08T20:11:25.883 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:25.869+0000 1694f700 -1 osd.6 213 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e211) 2019-12-08T20:11:26.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:26.174+0000 1694f700 -1 osd.0 214 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:26.997 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:26.956+0000 37792700 -1 osd.6 214 build_incremental_map_msg missing incremental map 212 2019-12-08T20:11:26.997 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:11:26.959+0000 37792700 -1 osd.6 214 build_incremental_map_msg missing incremental map 212 2019-12-08T20:11:27.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:11:27.197+0000 1694f700 -1 osd.0 214 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:110178 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5397336464 gen 1] snapc 0=[] ondisk+write+known_if_redirected e209) 2019-12-08T20:11:28.285 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:11:28.282+0000 20e25700 -1 RGWWatcher::handle_error cookie 5397336464 err (107) Transport endpoint is not connected 2019-12-08T20:12:47.567 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:47.560+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:48.019 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:48.014+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:48.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:48.148+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:48.214 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:48.209+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:48.325 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:48.317+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:48.549 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:48.544+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:48.925 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:48.919+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:49.017 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:49.012+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:49.139 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:49.134+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:49.206 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:49.202+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:49.321 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:49.318+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:49.487 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:49.483+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:49.512 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:49.507+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:49.965 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:49.960+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:49.988 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:49.984+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:50.174 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:50.170+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:50.254 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:50.251+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:50.372 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:50.368+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:50.453 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:50.447+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:50.498 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:50.494+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:50.951 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:50.945+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:50.953 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:50.948+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:51.149 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:51.145+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:51.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:51.265+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:51.348 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:51.343+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:51.492 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:51.487+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:51.495 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:51.491+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:51.932 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:51.922+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:51.970 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:51.964+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:52.140 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:52.137+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:52.291 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:52.288+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:52.326 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:52.323+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:52.500 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:52.497+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:52.539 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:52.531+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:52.985 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:52.980+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:52.997 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:52.993+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:53.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:53.142+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:53.284 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:53.279+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:53.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:53.298+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:53.467 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:53.463+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:53.584 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:53.580+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:53.963 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:53.959+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:54.015 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:54.010+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:54.127 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:54.123+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:54.271 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:54.268+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:54.283 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:54.279+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:54.442 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:54.438+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:54.620 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:54.616+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:54.985 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:54.979+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:55.041 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:55.031+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:55.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:55.117+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:55.265 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:55.261+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:55.288 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:55.283+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:55.410 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:55.405+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:55.641 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:55.632+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:56.026 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:56.021+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:56.059 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:56.040+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:56.059 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:56.050+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:12:56.143 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:56.139+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:56.252 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:56.248+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:56.335 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:56.330+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:56.397 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:56.392+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:56.648 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:56.642+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:56.981 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:56.975+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:57.046 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:57.041+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:57.048 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:57.044+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:12:57.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:57.193+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:57.247 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:57.243+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:57.318 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:57.314+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:57.385 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:57.380+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:57.680 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:57.674+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:57.996 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:57.990+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:58.071 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:58.065+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:58.072 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:58.066+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:12:58.172 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:58.163+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:58.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:58.202+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:58.349 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:58.344+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:58.409 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:58.405+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:58.733 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:58.727+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:58.999 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:58.993+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:12:59.124 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:59.118+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:12:59.124 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:12:59.119+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:12:59.214 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:12:59.209+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:12:59.251 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:12:59.248+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:12:59.361 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:12:59.358+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:12:59.440 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:12:59.437+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:12:59.735 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:12:59.728+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:12:59.975 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:12:59.969+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:00.175 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:00.169+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:00.176 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:00.170+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:00.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:00.228+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:00.257 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:00.254+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:00.404 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:00.401+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:00.463 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:00.460+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:00.716 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:00.710+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:01.027 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:01.021+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:01.177 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:01.172+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:01.179 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:01.174+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:01.216 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:01.211+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:01.216 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:01.211+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:01.279 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:29:32.707 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.283 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:29:32.711 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.298 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:29:32.723 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.307 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:29:32.900 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.310 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:29:32.735 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.313 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:29:32.905 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.368 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:01.365+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:01.471 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:01.468+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:01.536 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:29:32.964 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.633 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:29:33.226 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.659 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:29:33.085 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:13:01.767 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:01.762+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:02.077 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:02.071+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:02.214 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:02.209+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:02.216 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:02.211+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:02.227 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:02.223+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:02.242 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:02.239+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:02.350 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:02.345+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:02.511 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:02.508+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:02.722 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:02.716+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:03.057 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:03.051+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:03.178 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:03.172+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:03.180 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:03.175+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:03.192 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:03.188+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:03.291 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:03.286+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:03.344 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:03.339+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:03.563 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:03.559+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:03.726 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:03.720+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:04.037 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:04.031+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:04.211 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:04.205+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:04.211 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:04.206+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:04.231 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:04.227+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:04.276 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:04.272+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:04.356 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:04.352+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:04.586 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:04.582+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:04.728 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:04.721+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:05.055 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:05.048+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:05.197 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:05.188+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:05.222 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:05.215+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:05.223 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:05.217+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:05.331 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:05.323+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:05.357 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:05.353+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:05.610 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:05.606+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:05.749 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:05.743+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:06.065 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:06.059+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:06.222 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:06.219+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:06.258 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:06.252+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:06.258 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:06.253+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:06.290 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:06.286+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:06.352 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:06.348+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:06.640 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:06.636+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:06.736 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:06.730+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:07.032 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:07.026+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:07.251 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:07.247+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:07.266 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:07.262+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:07.277 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:07.272+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:07.279 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:07.275+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:07.309 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:07.306+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:07.670 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:07.666+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:07.735 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:07.722+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:08.008 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:07.998+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:08.211 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:08.207+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:08.245 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:08.242+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:08.271 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:08.265+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:08.274 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:08.269+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:08.346 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:08.342+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:08.675 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:08.671+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:08.700 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:08.694+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:09.027 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:09.020+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:09.193 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:09.189+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:09.243 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:09.237+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:09.244 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:09.238+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:09.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:09.285+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:09.393 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:09.388+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:09.646 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:09.642+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:09.753 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:09.746+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:09.996 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:09.989+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:10.209 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:10.205+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:10.273 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:10.263+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:10.278 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:10.268+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:10.309 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:10.301+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:10.374 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:10.370+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:10.631 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:10.627+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:10.787 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:10.781+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:10.967 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:10.961+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:11.248 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:11.240+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:11.312 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:11.306+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:11.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:11.309+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:11.344 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:11.338+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:11.413 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:11.409+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:11.619 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:11.615+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:11.809 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:11.802+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:11.932 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:11.927+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:12.244 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:12.239+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:12.353 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:12.349+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:12.369 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:12.361+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:12.370 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:12.362+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:12.441 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:12.436+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:12.663 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:12.659+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:12.833 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:12.828+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:12.946 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:12.940+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:13.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:13.258+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:13.355 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:13.350+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:13.380 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:13.374+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:13.381 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:13.375+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:13.442 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:13.438+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:13.680 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:13.676+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:13.829 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:13.824+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:13.952 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:13.946+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:14.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:14.297+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:14.386 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:14.382+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:14.409 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:14.402+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:14.410 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:14.404+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:14.496 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:14.492+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:14.637 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:14.631+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:14.885 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:14.878+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:14.934 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:14.929+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:15.351 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:15.346+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:15.396 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:15.390+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:15.397 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:15.391+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:15.429 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:15.425+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:15.466 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:15.462+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:15.594 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:15.585+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:15.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:15.841+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:15.910 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:15.901+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:16.402 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:16.398+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:16.402 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:16.398+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:16.447 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:16.442+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:16.448 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:16.444+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:16.484 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:16.481+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:16.548 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:16.545+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:16.882 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:16.876+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:16.949 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:16.943+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:17.403 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:17.397+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:17.446 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:17.442+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:17.470 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:17.464+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:17.471 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:17.465+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:17.532 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:17.528+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:17.548 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:17.544+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:17.854 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:17.848+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:17.962 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:17.957+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:18.410 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:18.406+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:18.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:18.408+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:18.474 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:18.468+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:18.483 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:18.472+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:18.543 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:18.538+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:18.581 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:18.578+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:18.828 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:18.823+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:18.933 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:18.927+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:19.386 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:19.382+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:19.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:19.449+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:19.458 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:19.451+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:19.458 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:19.452+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:19.570 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:19.566+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:19.626 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:19.622+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:19.869 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:19.863+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:19.947 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:19.942+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:20.425 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:20.421+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:20.426 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:20.420+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:20.426 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:20.421+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:20.464 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:20.460+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:20.608 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:20.604+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:20.624 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:20.619+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:20.906 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:20.899+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:20.991 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:20.985+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:21.140 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:13:21.135+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e219 v219) 2019-12-08T20:13:21.426 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:21.420+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:21.427 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:21.421+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:21.460 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:21.456+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:21.461 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:21.457+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:21.618 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:21.614+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:21.669 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:21.665+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:21.937 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:21.932+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:22.034 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:22.029+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:22.450 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:22.441+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:22.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:22.443+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:22.467 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:22.464+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:22.484 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:22.481+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:22.617 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:22.613+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:22.679 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:22.675+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:22.950 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:22.943+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:23.073 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:23.067+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:23.414 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:23.403+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:23.416 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:23.406+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:23.520 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:23.511+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:23.529 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:23.521+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:23.669 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:23.665+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:23.731 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:23.727+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:23.936 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:23.929+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:24.119 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:24.114+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:24.442 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:24.437+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:24.444 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:24.439+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:24.535 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:24.532+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:24.576 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:24.571+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:24.638 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:24.635+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:24.692 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:24.689+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:24.897 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:24.891+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:25.165 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:25.158+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:25.459 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:25.452+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:25.459 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:25.454+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:25.551 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:13:25.546+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.443845+0000 front 2019-12-08T20:12:04.472444+0000 (oldest deadline 2019-12-08T20:12:47.660213+0000) 2019-12-08T20:13:25.590 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:25.586+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:25.638 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:13:25.634+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.953717+0000 front 2019-12-08T20:12:04.946982+0000 (oldest deadline 2019-12-08T20:12:48.190680+0000) 2019-12-08T20:13:25.715 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:13:25.711+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.497825+0000 front 2019-12-08T20:12:04.443986+0000 (oldest deadline 2019-12-08T20:12:46.524421+0000) 2019-12-08T20:13:25.874 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:13:25.868+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.006859+0000 front 2019-12-08T20:12:05.256476+0000 (oldest deadline 2019-12-08T20:12:49.075476+0000) 2019-12-08T20:13:26.132 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:13:26.126+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:05.255692+0000 front 2019-12-08T20:12:05.253938+0000 (oldest deadline 2019-12-08T20:12:47.250051+0000) 2019-12-08T20:13:26.213 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:13:26.204+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e219 v219) 2019-12-08T20:13:26.489 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:26.484+0000 1694f700 -1 osd.4 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.024194+0000 front 2019-12-08T20:12:03.514599+0000 (oldest deadline 2019-12-08T20:12:48.901845+0000) 2019-12-08T20:13:26.489 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:26.484+0000 1694f700 -1 osd.4 219 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:13:26.608 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:13:26.596+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:12:04.022582+0000 front 2019-12-08T20:12:04.436263+0000 (oldest deadline 2019-12-08T20:12:47.795204+0000) 2019-12-08T20:13:27.519 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:13:27.514+0000 1694f700 -1 osd.4 220 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110504 2.25s0 2:a66b5c54:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ef%84%91%e8%94%8a%e5%98%86%ef%84%92%e1%a0%81%eb%8c%8b%e0%a4%80%ce%a9%ed%88%8d%eb%8c%8d%e8%94%8a%e3%9c%84%e1%a0%83%e0%a4%81%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7069 in=7069b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (58) in=78b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-08T20:14:00.414 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.390+0000 3578e700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.462 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.413+0000 32788700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.473 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.426+0000 35f8f700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.473 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.432+0000 31f87700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.493 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.451+0000 32f89700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.545 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.523+0000 3578e700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.616 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.589+0000 31f87700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:00.709 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:00.653+0000 32f89700 -1 osd.6 225 build_incremental_map_msg missing incremental map 225 2019-12-08T20:14:22.262 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:22.258+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:22.508 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:22.501+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:23.264 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:23.259+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:23.534 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:23.527+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:24.173 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:24.170+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:24.224 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:24.214+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:24.282 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:24.277+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:24.581 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:24.575+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:25.191 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:25.184+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:25.270 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:25.263+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:25.312 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:25.307+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:25.585 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:25.579+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:25.946 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:25.940+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:25.972 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:25.968+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:26.235 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:26.231+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:26.285 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:26.279+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:26.356 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:26.351+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:26.546 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:26.540+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:26.915 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:26.909+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:26.964 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:26.959+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:27.195 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:27.191+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:27.284 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:27.277+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:27.369 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:27.365+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:27.567 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:27.561+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:27.965 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:27.959+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:27.968 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:27.964+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:28.163 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:28.158+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:28.329 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:28.323+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:28.401 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:28.398+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:28.548 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:28.542+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:28.928 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:28.922+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:29.012 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:29.008+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:29.148 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:29.142+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:29.362 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:29.356+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:29.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:29.449+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:29.543 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:29.537+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:29.936 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:29.930+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:30.012 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:30.007+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:30.026 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:30.021+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:30.150 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:30.146+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:30.333 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:30.327+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:30.420 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:30.415+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:30.522 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:30.516+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:30.972 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:30.966+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:31.022 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:31.017+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:31.030 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:31.027+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:31.137 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:31.133+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:31.314 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:31.309+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:31.408 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:31.403+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:31.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:31.405+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:31.516 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:31.512+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:31.980 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:31.975+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:32.008 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:32.004+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:32.049 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:32.044+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:32.166 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:32.160+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:32.287 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:32.282+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:32.424 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:32.420+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:32.426 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:32.423+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:32.499 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:32.494+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:32.990 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:32.985+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:32.992 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:32.985+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:33.058 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:33.053+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:33.215 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:33.210+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:33.336 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:33.330+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:33.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:33.458+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:33.463 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:33.460+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:33.521 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:33.515+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:33.972 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:33.967+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:33.974 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:33.968+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:34.069 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:34.062+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:34.262 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:34.257+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:34.299 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:34.293+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:34.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:34.448+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:34.453 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:34.450+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:34.532 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:34.527+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:34.968 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:34.963+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:34.979 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:34.975+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:35.046 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:35.041+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:35.230 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:35.225+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:35.273 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:35.266+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:35.461 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:35.457+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:35.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:35.459+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:35.534 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:35.529+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:35.997 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:35.990+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:36.009 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:36.004+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:36.081 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:36.074+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:36.086 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:36.078+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:36.257 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:36.252+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:36.267 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:36.262+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:36.447 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:36.442+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:36.448 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:36.444+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:36.497 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:36.491+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:36.967 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:36.961+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:37.041 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:37.037+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:37.132 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:37.122+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:37.136 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:37.125+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:37.213 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:37.209+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:37.284 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:37.276+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:37.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:37.433+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:37.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:37.436+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:37.545 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:37.538+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:37.941 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:37.934+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:38.028 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:38.021+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:38.121 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:38.113+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:38.121 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:38.116+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:38.240 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:38.236+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:38.313 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:38.306+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:38.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:38.447+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:38.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:38.449+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:38.507 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:38.500+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:38.975 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:38.967+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:39.035 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:39.030+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:39.101 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:39.093+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:39.102 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:39.096+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:39.256 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:39.251+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:39.363 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:39.347+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:39.464 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:39.456+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:39.464 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:39.459+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:39.498 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:39.491+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:40.009 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:40.002+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:40.074 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:40.070+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:40.080 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:40.073+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:40.085 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:40.080+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:40.210 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:40.205+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:40.393 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:40.386+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:40.510 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:40.503+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:40.510 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:40.506+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:40.518 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:40.508+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:41.008 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:41.001+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:41.115 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:41.109+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:41.117 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:41.113+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:41.119 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:41.114+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:41.165 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:41.160+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:41.431 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:41.424+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:41.485 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:41.480+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:41.486 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:41.482+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:41.508 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:41.498+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:42.027 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:42.021+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:42.131 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:42.126+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:42.147 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:42.139+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:42.148 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:42.142+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:42.153 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:42.148+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:42.460 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:42.455+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:42.461 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:42.457+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:42.483 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:42.477+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:42.503 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:42.497+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:43.072 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:43.065+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:43.162 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:43.157+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:43.163 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:43.156+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:43.167 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:43.161+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:43.172 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:43.167+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:43.499 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:43.494+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:43.503 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:43.499+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:43.539 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:43.532+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:43.559 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:43.552+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:44.032 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:44.025+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:44.133 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:44.127+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:44.136 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:44.131+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:44.184 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:44.180+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:44.204 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:44.200+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:44.502 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:44.497+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:44.505 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:44.499+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:44.507 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:44.500+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:44.613 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:44.606+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:45.040 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:45.034+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:45.115 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:45.108+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:45.119 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:45.113+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:45.183 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:45.178+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:45.217 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:45.209+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:45.472 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:45.467+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:45.475 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:45.472+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:45.559 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:45.551+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:45.587 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:45.580+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:46.054 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:46.048+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:46.141 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:46.127+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:46.141 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:46.129+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:46.157 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:46.153+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:46.171 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:46.167+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:46.470 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:46.467+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:46.471 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:46.469+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:46.592 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:46.586+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:46.611 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:46.604+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:47.103 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:47.097+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:47.170 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:47.162+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:47.172 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:47.165+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:47.187 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:47.181+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:47.227 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:47.221+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:47.513 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:47.508+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:47.514 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:47.510+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:47.564 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:47.559+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:47.639 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:47.631+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:48.112 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:48.105+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:48.155 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:48.150+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:48.193 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:48.186+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:48.194 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:48.188+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:48.271 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:48.268+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:48.528 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:48.522+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:48.530 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:48.524+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:48.596 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:48.589+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:48.683 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:48.677+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:49.095 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:49.089+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:49.108 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:49.104+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:49.191 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:49.183+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:49.192 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:49.187+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:49.227 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:49.223+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:49.518 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:49.514+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:49.519 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:49.516+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:49.580 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:49.573+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:49.671 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:49.664+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:50.091 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:50.086+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:50.117 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:50.109+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:50.183 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:50.175+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:50.184 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:50.178+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:50.274 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:50.270+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:50.561 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:50.557+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:50.562 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:50.560+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:50.567 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:50.559+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:50.686 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:50.679+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:51.067 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:51.063+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:51.117 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:51.110+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:51.194 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:51.186+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:51.201 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:51.189+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:51.273 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:51.268+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:51.526 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:51.519+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:51.622 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:51.605+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:51.622 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:51.607+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:51.731 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:51.725+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:52.104 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:52.097+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:52.115 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:52.108+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:52.210 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:52.203+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:52.212 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:52.206+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:52.284 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:52.280+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:52.531 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:52.524+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:52.607 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:52.603+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:52.609 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:52.605+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:52.697 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:52.690+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:53.075 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:53.068+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:53.146 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:53.140+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:53.233 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:53.225+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:53.236 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:53.228+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:53.323 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:53.318+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:53.485 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:53.479+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:53.583 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:53.578+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:53.588 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:53.583+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:53.726 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:53.719+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:54.075 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:54.068+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:54.160 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:54.156+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:54.272 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:54.264+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:54.273 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:54.267+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:54.342 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:54.338+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:54.471 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:54.462+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:54.565 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:54.560+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:54.566 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:54.563+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:54.686 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:54.680+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:55.032 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:55.024+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:55.182 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:55.176+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:55.257 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:55.247+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:55.257 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:55.250+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:55.303 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:55.298+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:55.461 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:55.449+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:55.542 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:55.538+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:55.543 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:55.539+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:55.703 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:55.696+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:56.084 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:56.077+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:56.210 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:56.203+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:56.213 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:56.207+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:56.218 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:56.213+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:56.284 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:56.278+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:56.425 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:56.418+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:56.486 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:14:56.479+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e225 v225) 2019-12-08T20:14:56.520 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:56.515+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:56.521 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:56.517+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:56.756 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:56.749+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:57.128 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:57.121+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:57.195 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:57.187+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:57.196 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:57.190+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:57.221 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:57.217+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:57.292 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:57.287+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:57.475 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:57.468+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:57.533 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:57.530+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:57.536 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:57.533+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:57.740 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:57.733+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:58.111 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:58.104+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:58.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:58.214+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:58.239 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:58.231+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:58.240 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:58.234+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:58.322 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:58.316+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:58.450 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:58.444+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:58.552 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:58.544+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:58.552 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:58.548+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:58.775 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:58.768+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:14:59.097 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:14:59.089+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:14:59.241 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:14:59.235+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:14:59.276 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:59.263+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:14:59.277 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:14:59.268+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:14:59.287 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:14:59.283+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:14:59.458 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:14:59.450+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:14:59.540 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:59.533+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:14:59.540 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:14:59.536+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:14:59.816 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:14:59.810+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:00.105 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:00.099+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:00.197 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:00.191+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:00.245 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:00.237+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:00.245 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:00.240+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:00.328 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:00.318+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:00.494 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:00.487+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:00.510 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:00.505+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:00.513 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:00.509+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:00.832 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:00.826+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:01.106 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:01.099+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:01.238 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:01.230+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:01.243 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:01.233+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:01.245 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:01.239+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:01.327 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:01.323+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:01.488 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:01.482+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:01.489 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:01.484+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:01.492 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:15:01.488+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e225 v225) 2019-12-08T20:15:01.512 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:01.505+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:01.816 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:01.809+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:02.099 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:02.090+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:02.240 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:02.233+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:02.244 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:02.239+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:02.247 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:02.242+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:02.333 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:02.329+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:02.489 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:02.483+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:02.493 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:02.488+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:02.562 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:02.553+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:02.796 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:02.788+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:03.053 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:03.045+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:03.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:03.218+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:03.276 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:03.268+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:03.277 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:03.271+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:03.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:03.360+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:03.532 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:03.528+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:03.534 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:03.530+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:03.583 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:03.577+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:03.770 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:03.763+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:04.033 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:04.026+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:04.183 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:04.178+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:04.252 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:04.244+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:04.256 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:04.247+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:04.397 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:04.391+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:04.551 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:04.545+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:04.581 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:04.575+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:04.585 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:04.580+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:04.746 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:04.740+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:05.061 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:05.054+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:05.162 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:05.156+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:05.242 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:05.234+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:05.243 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:05.237+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:05.366 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:05.350+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:05.551 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:05.545+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:05.551 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:05.547+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:05.558 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:05.552+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:05.745 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:05.739+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:06.065 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:15:06.057+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:40.818099+0000 front 2019-12-08T20:13:44.210207+0000 (oldest deadline 2019-12-08T20:14:25.477559+0000) 2019-12-08T20:15:06.153 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:15:06.148+0000 1694f700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.642852+0000 front 2019-12-08T20:13:42.741890+0000 (oldest deadline 2019-12-08T20:14:25.460489+0000) 2019-12-08T20:15:06.231 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:06.225+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:13:55.092864+0000 (oldest deadline 2019-12-08T20:14:35.092864+0000) 2019-12-08T20:15:06.235 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:06.230+0000 1694f700 -1 osd.6 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:06.341 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:15:06.336+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.054257+0000 front 2019-12-08T20:13:41.055116+0000 (oldest deadline 2019-12-08T20:14:23.258527+0000) 2019-12-08T20:15:06.519 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:15:06.512+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.938155+0000 front 2019-12-08T20:13:41.842367+0000 (oldest deadline 2019-12-08T20:14:24.038586+0000) 2019-12-08T20:15:06.522 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:06.518+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:42.073216+0000 front 2019-12-08T20:13:42.104566+0000 (oldest deadline 2019-12-08T20:14:21.649399+0000) 2019-12-08T20:15:06.523 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:06.520+0000 1694f700 -1 osd.0 225 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:06.540 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:15:06.536+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e225 v225) 2019-12-08T20:15:06.750 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:15:06.745+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:13:41.857729+0000 front 2019-12-08T20:13:42.039108+0000 (oldest deadline 2019-12-08T20:14:21.773834+0000) 2019-12-08T20:15:07.272 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:07.260+0000 1694f700 -1 osd.6 226 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e223) 2019-12-08T20:15:07.575 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:15:07.572+0000 1694f700 -1 osd.0 226 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:110671 5.5 5:a93a5511:::notify.2:head [watch ping cookie 415662352 gen 2] snapc 0=[] ondisk+write+known_if_redirected e223) 2019-12-08T20:15:07.598 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:15:07.445+0000 3378a700 -1 osd.6 226 build_incremental_map_msg missing incremental map 225 2019-12-08T20:15:08.878 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:15:08.864+0000 20e25700 -1 RGWWatcher::handle_error cookie 415662352 err (107) Transport endpoint is not connected 2019-12-08T20:15:11.879 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:31:43.306 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:11.880 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:31:43.308 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:11.923 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:31:43.348 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:11.946 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:31:43.372 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:11.962 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:31:43.555 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:11.965 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:31:43.558 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:12.077 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:31:43.494 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:12.230 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:31:43.656 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:15:12.247 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:31:43.840 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.126 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:33:03.554 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.165 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:33:03.591 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.172 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:33:03.600 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.184 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:33:03.610 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.210 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:33:03.803 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.215 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:33:03.808 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.254 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:33:03.682 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.256 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:33:03.682 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:32.299 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:33:03.891 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:34.942 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:34.935+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:35.008 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:35.003+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:35.706 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:35.700+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:35.762 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:35.757+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:35.881 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:35.873+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:35.964 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:35.958+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:36.027 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:36.021+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:36.060 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:36.054+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:36.691 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:36.686+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:36.729 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:36.724+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:36.870 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:36.865+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:36.937 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:36.932+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:36.984 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:36.978+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:37.020 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:37.014+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:37.718 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:37.715+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:37.735 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:37.731+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:37.912 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:37.908+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:37.961 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:37.956+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:37.976 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:37.969+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:38.040 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:38.035+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:38.417 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:38.412+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:38.707 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:38.702+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:38.771 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:38.767+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:38.926 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:38.923+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:38.939 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:38.934+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:38.996 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:38.992+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:39.080 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:39.074+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:39.426 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:39.421+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:39.677 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:39.673+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:39.750 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:39.742+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:39.917 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:39.913+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:39.959 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:39.951+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:39.963 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:39.957+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:40.122 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:40.116+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:40.389 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:40.381+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:40.698 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:40.694+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:40.740 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:40.736+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:40.917 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:40.913+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:40.936 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:40.931+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:40.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:40.948+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:41.156 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:41.151+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:41.385 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:41.378+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:41.713 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:41.706+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:41.774 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:41.771+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:41.920 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:41.914+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:41.968 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:41.964+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:41.985 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:41.982+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:42.206 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:42.200+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:42.365 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:42.360+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:42.751 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:42.747+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:42.757 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:42.751+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:42.906 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:42.899+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:42.986 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:42.982+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:43.034 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:43.030+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:43.247 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:43.242+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:43.364 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:43.357+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:43.732 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:43.728+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:43.789 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:43.770+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:43.925 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:43.916+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:44.003 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:43.999+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:44.022 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:44.018+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:44.227 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:44.221+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:44.365 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:44.359+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:44.686 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:44.683+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:44.787 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:44.782+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:44.940 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:44.934+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:45.022 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:45.016+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:45.030 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:45.026+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:45.238 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:45.232+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:45.349 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:45.344+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:45.643 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:45.639+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:45.758 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:45.755+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:45.969 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:45.964+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:46.026 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:46.022+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:46.062 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:46.058+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:46.228 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:46.222+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:46.393 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:46.380+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:46.678 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:46.674+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:46.797 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:46.786+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:46.973 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:46.967+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:47.003 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:46.999+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:47.101 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:47.097+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:47.227 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:47.221+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:47.363 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:47.357+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:47.692 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:47.689+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:47.789 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:47.785+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:47.936 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:47.930+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:48.011 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:48.007+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:48.120 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:48.116+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:48.231 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:48.225+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:48.387 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:48.381+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:48.689 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:48.686+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:48.743 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:48.739+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:48.947 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:48.940+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:48.989 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:48.984+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:49.157 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:49.153+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:49.212 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:49.206+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:49.350 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:49.341+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:49.666 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:49.662+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:49.718 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:49.714+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:49.951 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:49.947+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:49.953 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:49.947+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:50.191 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:50.184+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:50.193 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:50.189+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:50.367 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:50.360+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:50.658 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:50.654+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:50.704 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:50.700+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:50.926 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:50.920+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:50.999 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:50.995+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:51.206 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:51.202+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:51.220 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:51.213+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:51.345 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:51.338+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:51.614 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:51.610+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:51.741 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:51.734+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:51.960 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:51.954+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:52.036 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:52.031+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:52.248 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:52.244+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:52.249 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:52.240+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:52.322 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:52.316+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:52.345 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:33:23.772 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.354 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:33:23.781 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.359 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:33:23.785 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.363 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:33:23.789 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.372 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:33:23.965 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.383 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:33:23.976 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.405 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:33:23.833 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.411 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:33:23.837 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.459 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:33:24.052 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:16:52.618 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:52.615+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:52.705 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:52.702+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:52.991 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:52.985+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:53.076 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:53.072+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:53.260 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:53.251+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:53.261 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:53.254+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:53.299 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:53.295+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:53.345 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:53.340+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:53.653 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:53.649+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:53.751 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:53.747+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:54.025 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:54.019+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:54.120 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:54.114+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:54.244 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:54.238+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:54.246 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:54.240+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:54.332 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:54.328+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:54.348 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:54.343+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:54.676 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:54.672+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:54.757 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:54.753+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:55.052 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:55.048+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:55.089 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:55.084+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:55.227 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:55.222+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:55.229 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:55.225+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:55.305 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:55.300+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:55.367 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:55.362+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:55.635 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:55.631+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:55.726 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:55.721+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:56.047 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:56.041+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:56.087 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:56.081+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:56.246 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:56.241+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:56.246 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:56.241+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:56.324 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:56.320+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:56.337 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:56.332+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:56.630 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:56.625+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:56.735 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:56.731+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:57.090 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:57.086+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:57.093 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:57.087+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:57.208 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:57.196+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:57.209 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:57.197+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:57.347 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:57.342+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:57.348 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:57.344+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:57.657 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:57.653+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:57.776 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:57.771+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:58.074 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:58.068+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:58.092 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:58.088+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:58.163 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:58.157+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:58.164 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:58.159+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:58.322 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:58.319+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:58.356 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:58.351+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:58.631 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:58.625+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:58.753 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:58.749+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:16:59.097 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:16:59.092+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:16:59.136 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:16:59.133+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:16:59.218 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:59.212+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:16:59.221 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:16:59.213+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:16:59.336 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:16:59.332+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:16:59.350 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:16:59.346+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:16:59.611 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:16:59.606+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:16:59.708 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:16:59.704+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:00.081 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:00.076+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:00.158 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:00.153+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:00.245 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:00.239+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:00.245 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:00.241+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:00.376 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:00.370+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:00.384 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:00.379+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:00.564 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:00.560+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:00.668 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:00.665+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:01.108 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:01.102+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:01.119 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:01.115+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:01.244 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:01.239+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:01.246 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:01.243+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:01.404 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:01.400+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:01.406 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:01.402+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:01.557 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:01.552+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:01.678 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:01.674+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:02.099 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:02.093+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:02.170 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:02.166+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:02.208 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:02.202+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:02.208 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:02.204+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:02.419 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:02.415+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:02.435 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:02.432+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:02.577 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:02.573+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:02.703 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:02.700+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:03.098 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:03.093+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:03.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:03.135+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:03.236 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:03.230+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:03.236 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:03.232+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:03.376 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:03.371+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:03.451 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:03.447+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:03.564 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:03.560+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:03.723 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:03.719+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:04.131 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:04.126+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:04.172 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:04.168+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:04.274 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:04.270+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:04.276 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:04.272+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:04.395 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:04.390+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:04.441 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:04.437+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:04.603 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:04.599+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:04.776 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:04.772+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:05.169 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:05.163+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:05.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:05.195+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:05.312 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:05.307+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:05.316 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:05.308+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:05.399 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:05.395+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:05.415 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:05.409+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:05.583 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:05.579+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:05.810 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:05.806+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:06.178 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:06.174+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:06.178 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:06.173+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:06.357 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:06.350+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:06.360 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:06.353+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:06.384 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:06.380+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:06.448 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:06.443+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:06.632 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:06.628+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:06.787 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:06.782+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:07.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:07.131+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:07.184 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:07.179+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:07.210 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:17:07.204+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 44sec e230 v230) 2019-12-08T20:17:07.373 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:07.369+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:07.394 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:07.389+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:07.396 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:07.392+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:07.436 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:07.431+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:07.622 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:07.618+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:07.789 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:07.786+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:08.136 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:08.132+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:08.157 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:08.152+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:08.410 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:08.406+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:08.439 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:08.434+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:08.439 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:08.435+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:08.443 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:08.439+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:08.634 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:08.630+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:08.831 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:08.826+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:09.150 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:09.146+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:09.185 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:09.180+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:09.439 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:09.435+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:09.447 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:09.442+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:09.447 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:09.443+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:09.454 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:09.450+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:09.607 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:09.603+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:09.881 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:09.878+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:10.134 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:10.127+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:10.196 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:10.192+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:10.410 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:10.406+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:10.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:10.425+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:10.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:10.426+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:10.477 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:10.471+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:10.603 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:10.600+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:10.877 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:10.874+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:11.134 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:11.130+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:11.241 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:11.235+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:11.410 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:11.405+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:11.412 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:11.407+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:11.455 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:11.451+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:11.484 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:17:11.479+0000 1694f700 -1 osd.7 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:55.984954+0000 front 2019-12-08T20:15:53.569998+0000 (oldest deadline 2019-12-08T20:16:38.062046+0000) 2019-12-08T20:17:11.643 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:17:11.637+0000 1694f700 -1 osd.0 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:51.356989+0000 front 2019-12-08T20:15:50.847774+0000 (oldest deadline 2019-12-08T20:16:34.955134+0000) 2019-12-08T20:17:11.907 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:17:11.903+0000 1694f700 -1 osd.2 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.715727+0000 front 2019-12-08T20:15:50.846452+0000 (oldest deadline 2019-12-08T20:16:35.564968+0000) 2019-12-08T20:17:12.160 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:17:12.156+0000 1694f700 -1 osd.3 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:50.847223+0000 front 2019-12-08T20:15:50.709857+0000 (oldest deadline 2019-12-08T20:16:34.399652+0000) 2019-12-08T20:17:12.276 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:17:12.268+0000 1694f700 -1 osd.5 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.756990+0000 front 2019-12-08T20:15:53.693727+0000 (oldest deadline 2019-12-08T20:16:35.225650+0000) 2019-12-08T20:17:12.289 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:17:12.283+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 44sec e230 v230) 2019-12-08T20:17:12.387 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:12.381+0000 1694f700 -1 osd.4 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:53.693343+0000 front 2019-12-08T20:15:52.939737+0000 (oldest deadline 2019-12-08T20:16:34.013318+0000) 2019-12-08T20:17:12.388 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:12.383+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:12.453 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:17:12.450+0000 1694f700 -1 osd.1 230 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:15:52.191083+0000 front 2019-12-08T20:15:53.692595+0000 (oldest deadline 2019-12-08T20:16:35.627908+0000) 2019-12-08T20:17:13.383 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:13.378+0000 1694f700 -1 osd.4 231 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:14.390 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:17:14.385+0000 1694f700 -1 osd.4 232 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111000 2.25s0 2:a479988e:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%ed%88%8d%ed%88%8d%e5%98%86%e0%a4%80%e0%a4%80%e1%a0%82%e0%a4%83%e0%a4%81%ef%84%92%e9%90%8c%e3%9c%85%e1%a0%81%e5%98%87%e5%98%88%eb%8c%8d:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~22070 in=22070b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e230) 2019-12-08T20:17:45.853 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:17:45.693+0000 34f8d700 -1 osd.6 235 build_incremental_map_msg missing incremental map 235 2019-12-08T20:17:45.885 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:17:45.695+0000 30f85700 -1 osd.6 235 build_incremental_map_msg missing incremental map 235 2019-12-08T20:17:46.141 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:17:46.126+0000 3378a700 -1 osd.6 235 build_incremental_map_msg missing incremental map 235 2019-12-08T20:17:46.189 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:17:46.181+0000 37792700 -1 osd.6 235 build_incremental_map_msg missing incremental map 235 2019-12-08T20:18:03.168 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:03.164+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:04.184 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:04.179+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:05.161 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:05.157+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:06.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:06.133+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:07.094 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:07.090+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:08.114 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:08.110+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:09.143 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:09.139+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:10.185 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:10.180+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:10.562 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:10.557+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:11.223 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:11.220+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:11.457 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:11.448+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:11.546 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:11.538+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:12.244 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:12.240+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:12.453 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:12.449+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:12.566 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:12.560+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:13.011 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:13.004+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:13.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:13.228+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:13.505 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:13.500+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:13.604 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:13.597+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:14.020 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:14.012+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:14.188 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:14.185+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:14.253 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:14.249+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:14.502 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:14.498+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:14.614 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:14.608+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:14.744 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:14.736+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:15.068 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:15.061+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:15.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:15.210+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:15.217 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:15.212+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:15.219 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:15.215+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:15.499 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:15.494+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:15.653 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:15.639+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:15.757 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:15.751+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:16.043 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:16.033+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:16.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:16.187+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:16.194 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:16.191+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:16.268 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:16.262+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:16.499 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:16.494+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:16.656 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:16.648+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:16.723 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:16.718+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:16.751 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:16.744+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:16.997 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:16.990+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:17.166 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:17.160+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:17.167 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:17.163+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:17.233 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:17.229+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:17.540 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:17.534+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:17.684 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:17.678+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:17.693 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:17.689+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:17.803 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:17.797+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:17.980 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:17.971+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:18.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:18.116+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:18.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:18.117+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:18.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:18.220+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:18.531 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:18.527+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:18.670 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:18.665+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:18.720 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:18.715+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:18.836 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:18.830+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:18.975 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:18.967+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:19.079 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:19.073+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:19.080 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:19.076+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:19.221 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:19.216+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:19.503 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:19.492+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:19.650 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:19.643+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:19.769 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:19.763+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:19.805 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:19.798+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:19.935 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:19.929+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:20.048 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:20.043+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:20.048 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:20.045+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:20.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:20.219+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:20.547 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:20.543+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:20.636 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:20.630+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:20.797 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:20.791+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:20.835 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:20.827+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:20.904 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:20.897+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:21.093 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:21.088+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:21.096 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:21.092+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:21.246 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:21.242+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:21.562 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:21.557+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:21.634 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:21.627+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:21.793 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:21.787+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:21.827 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:21.820+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:21.941 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:21.934+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:22.051 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:22.045+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:22.052 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:22.049+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:22.231 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:22.226+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:22.543 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:22.536+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:22.634 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:22.627+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:22.808 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:22.800+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:22.809 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:22.803+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:22.854 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:22.846+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:22.958 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:22.951+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:23.065 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:23.059+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:23.066 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:23.062+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:23.219 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:23.214+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:23.575 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:23.569+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:23.611 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:23.604+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:23.820 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:23.812+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:23.850 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:23.843+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:23.851 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:23.846+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:23.944 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:23.938+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:24.026 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:24.021+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:24.027 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:24.023+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:24.229 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:24.224+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:24.529 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:24.525+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:24.653 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:24.646+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:24.815 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:24.806+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:24.817 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:24.810+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:24.841 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:24.835+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:24.958 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:24.951+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:25.060 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:25.055+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:25.063 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:25.058+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:25.253 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:25.247+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:25.486 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:25.481+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:25.658 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:25.652+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:25.803 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:25.795+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:25.804 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:25.798+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:25.836 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:25.829+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:25.920 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:25.913+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:26.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:26.069+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:26.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:26.074+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:26.304 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:26.299+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:26.478 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:26.473+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:26.631 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:26.624+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:26.786 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:26.778+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:26.787 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:26.781+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:26.844 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:26.837+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:26.883 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:26.876+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:27.044 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:27.038+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:27.045 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:27.041+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:27.341 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:27.337+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:27.522 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:27.518+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:27.620 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:27.612+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:27.823 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:27.816+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:27.825 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:27.819+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:27.833 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:27.825+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:27.920 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:27.914+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:28.043 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:28.037+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:28.043 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:28.040+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:28.350 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:28.347+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:28.483 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:28.478+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:28.638 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:28.632+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:28.829 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:28.818+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:28.842 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:28.836+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:28.845 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:28.841+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:28.893 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:28.886+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:29.003 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:28.998+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:29.004 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:29.000+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:29.329 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:29.324+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:29.461 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:29.456+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:29.683 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:29.674+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:29.832 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:29.824+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:29.833 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:29.827+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:29.869 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:29.860+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:29.892 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:29.871+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:30.028 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:30.022+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:30.029 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:30.025+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:30.321 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:30.316+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:30.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:30.455+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:30.735 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:30.728+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:30.825 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:30.817+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:30.826 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:30.820+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:30.877 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:30.871+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:30.919 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:30.913+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:31.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:30.999+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:31.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:31.002+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:31.369 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:31.364+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:31.446 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:31.443+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:31.759 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:31.751+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:31.860 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:31.853+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:31.862 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:31.856+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:31.878 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:31.872+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:31.880 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:31.873+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:31.960 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:31.954+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:31.961 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:31.957+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:32.406 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:32.400+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:32.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:32.455+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:32.780 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:32.773+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:32.834 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:32.827+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:32.838 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:32.832+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:32.874 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:32.868+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:32.883 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:32.876+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:32.977 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:32.971+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:32.977 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:32.974+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:33.406 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:33.400+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:33.420 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:33.416+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:33.800 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:33.795+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:33.823 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:33.816+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:33.823 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:33.819+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:33.919 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:33.912+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:33.927 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:33.919+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:34.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:34.007+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:34.015 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:34.012+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:34.368 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:34.364+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:34.382 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:34.378+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:34.784 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:34.777+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:34.820 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:34.813+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:34.821 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:34.815+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:34.938 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:34.932+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:34.968 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:34.961+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:35.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:35.037+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:35.045 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:35.042+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:35.358 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:35.352+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:35.374 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:35.369+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:35.776 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:35.769+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:35.832 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:35.825+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:35.833 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:35.828+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:35.936 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:35.930+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:35.984 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:35.976+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:36.029 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:36.025+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:36.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:36.029+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:36.332 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:36.327+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:36.376 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:36.371+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:36.741 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:36.734+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:36.807 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:36.799+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:36.808 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:36.803+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:36.938 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:36.931+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:37.006 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:36.999+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:37.054 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:37.050+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:37.057 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:37.053+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:37.340 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:37.335+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:37.372 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:37.367+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:37.753 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:37.744+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:37.766 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:37.758+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:37.767 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:37.761+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:37.961 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:37.954+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:37.981 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:37.974+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:38.036 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:38.033+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:38.038 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:38.036+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:38.326 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:38.321+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:38.374 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:38.370+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:38.778 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:38.771+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:38.779 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:38.773+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:38.783 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:38.776+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:38.931 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:38.923+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:38.962 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:38.955+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:39.010 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:39.005+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:39.011 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:39.007+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:39.336 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:39.331+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:39.411 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:39.407+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:39.744 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:39.739+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:39.748 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:39.744+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:39.789 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:39.783+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:39.951 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:39.945+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:39.981 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:39.971+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:40.044 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:40.038+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:40.048 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:40.044+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:40.343 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:40.334+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:40.409 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:40.406+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:40.773 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:40.768+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:40.777 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:40.773+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:40.793 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:40.788+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:40.946 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:40.941+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:40.973 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:40.966+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:41.015 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:41.009+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:41.016 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:41.012+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:41.296 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:41.292+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:41.371 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:41.366+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:41.750 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:41.743+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:41.751 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:41.746+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:41.781 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:41.774+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:41.952 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:41.946+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:41.976 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:41.969+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:42.025 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:42.020+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:42.028 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:42.024+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:42.345 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:42.339+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:42.350 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:42.344+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:42.571 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:18:42.565+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e236 v236) 2019-12-08T20:18:42.710 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:42.705+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:42.714 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:42.710+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:42.825 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:42.820+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:42.906 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:42.900+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:42.992 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:42.987+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:43.000 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:42.993+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:43.001 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:42.997+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:43.321 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:43.317+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:43.343 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:43.339+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:43.695 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:43.690+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:43.699 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:43.695+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:43.838 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:43.832+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:43.921 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:43.914+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:43.951 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:43.945+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:43.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:43.969+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:43.976 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:43.972+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:44.294 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:44.291+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:44.350 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:44.347+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:44.675 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:44.670+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:44.679 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:44.675+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:44.833 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:44.824+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:44.898 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:44.889+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:44.942 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:44.937+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:44.991 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:44.984+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:44.993 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:44.988+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:45.283 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:45.278+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:45.331 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:45.318+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:45.731 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:45.724+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:45.732 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:45.727+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:45.823 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:45.817+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:45.871 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:45.865+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:45.942 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:45.929+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:45.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:45.974+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:45.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:45.976+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:46.277 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:46.273+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:46.311 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:46.306+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:46.711 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:46.705+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:46.715 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:46.711+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:46.826 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:46.820+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:46.910 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:46.904+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:46.917 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:46.911+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:46.949 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:46.944+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:46.950 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:46.945+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:47.305 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:47.299+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:47.319 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:47.315+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:47.588 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:18:47.582+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e236 v236) 2019-12-08T20:18:47.711 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:47.707+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:47.714 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:47.710+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:47.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:47.873+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:47.907 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:47.901+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:47.915 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:47.909+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:47.988 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:47.983+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:47.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:47.986+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:48.275 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:48.269+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:48.312 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:48.308+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:48.764 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:48.759+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:48.768 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:48.764+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:48.897 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:48.884+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:48.897 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:48.891+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:48.929 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:48.924+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:49.044 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:49.037+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:49.049 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:49.044+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:49.272 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:49.265+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:49.331 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:49.326+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:49.818 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:49.813+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:49.822 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:49.817+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:49.902 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:49.895+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:49.947 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:49.941+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:49.980 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:49.973+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:50.062 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:50.052+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:50.064 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:50.055+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:50.230 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:50.225+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:50.340 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:50.335+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:50.805 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:50.797+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:50.806 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:50.800+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:50.933 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:50.927+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:50.956 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:50.950+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:51.014 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:51.009+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:51.110 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:51.103+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:51.110 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:51.106+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:51.217 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:51.211+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:51.349 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:51.344+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:51.785 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:51.778+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:51.786 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:51.781+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:51.972 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:18:51.965+0000 1694f700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.930609+0000 front 2019-12-08T20:17:29.851428+0000 (oldest deadline 2019-12-08T20:18:12.732494+0000) 2019-12-08T20:18:52.002 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:18:51.996+0000 1694f700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:25.975782+0000 front 2019-12-08T20:17:25.971129+0000 (oldest deadline 2019-12-08T20:18:10.040581+0000) 2019-12-08T20:18:52.060 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:18:52.055+0000 1694f700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.216147+0000 front 2019-12-08T20:17:29.231930+0000 (oldest deadline 2019-12-08T20:18:13.789087+0000) 2019-12-08T20:18:52.092 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:52.083+0000 1694f700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.847809+0000 front 2019-12-08T20:17:29.834101+0000 (oldest deadline 2019-12-08T20:18:14.512565+0000) 2019-12-08T20:18:52.096 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:52.087+0000 1694f700 -1 osd.0 236 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:52.204 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:18:52.197+0000 1694f700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:26.240067+0000 front 2019-12-08T20:17:26.492929+0000 (oldest deadline 2019-12-08T20:18:10.921794+0000) 2019-12-08T20:18:52.315 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:18:52.310+0000 1694f700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:17:29.736198+0000 front 2019-12-08T20:17:29.748293+0000 (oldest deadline 2019-12-08T20:18:13.802165+0000) 2019-12-08T20:18:52.626 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:18:52.621+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e236 v236) 2019-12-08T20:18:52.783 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:52.777+0000 1694f700 -1 osd.6 236 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:17:42.135910+0000 (oldest deadline 2019-12-08T20:18:22.135910+0000) 2019-12-08T20:18:52.784 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:18:52.780+0000 1694f700 -1 osd.6 236 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e234) 2019-12-08T20:18:53.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:53.162+0000 1694f700 -1 osd.0 237 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:54.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:18:54.203+0000 1694f700 -1 osd.0 238 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:111113 5.5 5:a93a5511:::notify.2:head [watch ping cookie 362188736 gen 1] snapc 0=[] ondisk+write+known_if_redirected e232) 2019-12-08T20:18:55.055 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:18:55.047+0000 20e25700 -1 RGWWatcher::handle_error cookie 362188736 err (107) Transport endpoint is not connected 2019-12-08T20:19:22.820 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:19:22.814+0000 11caf700 -1 osd.1 242 build_incremental_map_msg missing incremental map 242 2019-12-08T20:19:22.844 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:19:22.839+0000 11caf700 -1 osd.1 242 build_incremental_map_msg missing incremental map 242 2019-12-08T20:19:22.976 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:19:22.868+0000 30f85700 -1 osd.1 242 build_incremental_map_msg missing incremental map 242 2019-12-08T20:19:25.899 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:19:25.896+0000 3378a700 -1 osd.1 242 build_incremental_map_msg missing incremental map 242 2019-12-08T20:19:55.575 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:19:55.570+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:19:56.616 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:19:56.608+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:19:57.600 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:19:57.594+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:19:58.564 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:19:58.558+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:19:59.591 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:19:59.586+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:00.580 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:00.573+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:01.538 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:01.532+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:02.498 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:02.491+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:03.536 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:03.532+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:04.563 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:04.558+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:05.580 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:05.575+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:06.602 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:06.597+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:07.608 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:07.603+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:08.618 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:08.613+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:09.617 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:09.612+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:10.647 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:10.641+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:11.668 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:11.663+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:12.667 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:12.662+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:13.700 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:13.696+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:14.697 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:14.692+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:15.701 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:15.696+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:16.761 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:16.756+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:17.717 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:17.711+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:18.482 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:18.470+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:18.723 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:18.717+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:19.135 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:19.130+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:19.485 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:19.480+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:19.710 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:19.703+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:19.710 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:19.705+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:19.868 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:19.862+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:20.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:20.135+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:20.203 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:20.198+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:20.392 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:20.386+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:20.535 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:20.530+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:20.762 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:20.756+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:20.767 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:20.761+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:20.851 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:20.844+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:21.104 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:21.099+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:21.223 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:21.218+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:21.433 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:21.428+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:21.532 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:21.526+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:21.730 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:21.723+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:21.730 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:21.724+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:21.844 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:21.839+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:22.060 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:22.056+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:22.107 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:22.091+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:22.185 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:22.176+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:22.458 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:22.452+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:22.545 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:22.541+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:22.760 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:22.755+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:22.763 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:22.759+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:22.890 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:22.885+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:23.057 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:23.051+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:23.079 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:23.074+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:23.151 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:23.145+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:23.457 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:23.451+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:23.521 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:23.517+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:23.798 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:23.794+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:23.799 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:23.795+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:23.919 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:23.915+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:24.012 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:24.006+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:24.040 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:24.035+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:24.173 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:24.167+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:24.417 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:24.413+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:24.543 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:24.539+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:24.765 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:24.760+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:24.767 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:24.764+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:24.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:24.953+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:25.021 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:25.016+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:25.072 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:25.067+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:25.132 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:25.128+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:25.464 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:25.459+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:25.511 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:25.506+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:25.736 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:25.732+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:25.738 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:25.734+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:25.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:25.984+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:25.996 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:25.992+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:26.084 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:26.081+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:26.096 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:26.093+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:26.505 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:26.501+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:26.523 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:26.519+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:26.783 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:26.778+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:26.784 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:26.780+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:26.964 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:26.960+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:27.032 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:27.027+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:27.056 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:27.052+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:27.079 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:27.074+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:27.500 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:27.497+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:27.515 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:27.510+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:27.795 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:27.791+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:27.797 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:27.792+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:27.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:27.942+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:28.041 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:28.036+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:28.072 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:28.068+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:28.125 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:28.121+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:28.469 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:28.460+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:28.545 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:28.541+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:28.789 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:28.784+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:28.790 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:28.785+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:28.992 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:28.988+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:29.040 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:29.035+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:29.074 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:29.070+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:29.116 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:29.112+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:29.443 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:29.440+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:29.504 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:29.499+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:29.834 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:29.828+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:29.834 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:29.830+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:29.955 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:29.951+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:30.040 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:30.035+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:30.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:30.068+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:30.116 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:30.112+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:30.482 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:30.479+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:30.549 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:30.544+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:30.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:30.788+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:30.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:30.788+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:30.998 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:30.994+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:31.042 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:31.037+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:31.111 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:31.106+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:31.181 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:31.162+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:31.470 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:31.466+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:31.536 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:31.530+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:31.765 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:31.760+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:31.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:31.761+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:32.025 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:32.021+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:32.076 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:32.071+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:32.083 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:32.078+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:32.197 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:32.193+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:32.428 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:32.423+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:32.564 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:32.559+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:32.768 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:32.762+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:32.769 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:32.763+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:33.047 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:33.043+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:33.052 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:33.048+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:33.054 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:33.050+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:33.221 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:33.216+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:33.380 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:33.376+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:33.617 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:33.612+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:33.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:33.760+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:33.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:33.762+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:34.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:34.076+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:34.084 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:34.078+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:34.093 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:34.088+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:34.190 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:34.184+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:34.356 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:34.352+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:34.584 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:34.574+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:34.735 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:34.728+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:34.735 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:34.729+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:35.059 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:35.055+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:35.071 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:35.068+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:35.087 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:35.083+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:35.182 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:35.176+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:35.368 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:35.363+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:35.630 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:35.625+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:35.721 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:35.716+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:35.722 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:35.717+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:36.052 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:36.048+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:36.053 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:36.048+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:36.092 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:36.078+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:36.150 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:36.144+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:36.403 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:36.399+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:36.659 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:36.655+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:36.770 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:36.763+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:36.770 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:36.764+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:37.013 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:37.008+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:37.039 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:37.034+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:37.057 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:37.050+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:37.176 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:37.170+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:37.377 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:37.373+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:37.618 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:37.611+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:37.730 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:37.725+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:37.733 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:37.727+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:38.034 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:38.031+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:38.036 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:38.032+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:38.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:38.038+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:38.185 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:38.179+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:38.390 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:38.385+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:38.618 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:38.612+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:38.692 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:38.686+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:38.692 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:38.687+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:39.001 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:38.996+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:39.015 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:39.011+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:39.094 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:39.091+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:39.182 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:39.176+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:39.245 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:37:10.672 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.249 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:37:10.677 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.256 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:37:10.682 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.262 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:37:10.688 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.267 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:37:10.859 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.273 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:37:10.866 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.383 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:39.379+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:39.530 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:37:10.956 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.531 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:37:10.959 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.544 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:37:11.137 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:20:39.642 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:39.636+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:39.732 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:39.725+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:39.732 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:39.727+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:39.957 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:39.951+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:40.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:40.029+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:40.092 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:40.089+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:40.210 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:40.205+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:40.404 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:40.401+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:40.647 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:40.641+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:40.745 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:40.740+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:40.745 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:40.740+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:40.941 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:40.938+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:41.002 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:40.999+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:41.063 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:41.059+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:41.218 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:41.211+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:41.386 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:41.383+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:41.689 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:41.683+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:41.785 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:41.781+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:41.787 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:41.782+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:41.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:41.945+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:41.998 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:41.994+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:42.026 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:42.021+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:42.249 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:42.243+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:42.363 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:42.359+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:42.655 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:42.649+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:42.811 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:42.805+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:42.812 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:42.807+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:42.935 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:42.931+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:42.958 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:42.954+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:43.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:43.070+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:43.208 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:43.203+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:43.353 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:43.349+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:43.647 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:43.641+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:43.783 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:43.777+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:43.784 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:43.778+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:43.911 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:43.907+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:43.947 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:43.944+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:44.090 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:44.085+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:44.192 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:44.186+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:44.386 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:44.382+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:44.669 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:44.664+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:44.791 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:44.786+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:44.793 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:44.789+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:44.913 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:44.910+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:44.928 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:44.923+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:45.130 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:45.126+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:45.204 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:45.199+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:45.386 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:45.381+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:45.647 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:45.641+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:45.846 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:45.840+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:45.846 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:45.842+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:45.962 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:45.959+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:45.966 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:45.963+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:46.109 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:46.105+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:46.245 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:46.239+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:46.428 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:46.425+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:46.682 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:46.676+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:46.803 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:46.798+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:46.805 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:46.801+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:46.945 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:46.941+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:46.945 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:46.942+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:47.144 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:47.139+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:47.224 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:47.212+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:47.452 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:47.448+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:47.685 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:47.679+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:47.761 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:47.756+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:47.762 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:47.757+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:47.945 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:47.941+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:47.975 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:47.972+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:48.102 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:48.099+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:48.215 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:48.208+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:48.422 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:48.417+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:48.716 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:48.710+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:48.736 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:48.730+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:48.736 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:48.731+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:48.983 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:48.979+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:48.996 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:48.992+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:49.126 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:49.122+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:49.253 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:49.246+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:49.456 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:49.453+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:49.712 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:49.706+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:49.713 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:49.707+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:49.750 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:49.744+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:49.959 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:49.955+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:50.033 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:49.989+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:50.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:50.169+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:50.233 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:50.228+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:50.486 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:50.482+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:50.722 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:50.716+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:50.731 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:50.726+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:50.733 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:50.729+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:50.956 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:50.952+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:50.991 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:50.987+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:51.174 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:51.169+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:51.222 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:51.216+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:51.510 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:51.506+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:51.712 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:51.706+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:51.713 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:51.707+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:51.747 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:51.741+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:51.970 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:51.966+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:51.998 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:51.994+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:52.167 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:20:52.163+0000 1694f700 -1 osd.0 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.004604+0000 front 2019-12-08T20:19:34.017754+0000 (oldest deadline 2019-12-08T20:20:19.835738+0000) 2019-12-08T20:20:52.211 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:52.205+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:52.526 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:20:52.522+0000 1694f700 -1 osd.1 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.750201+0000 front 2019-12-08T20:19:38.079202+0000 (oldest deadline 2019-12-08T20:20:17.652020+0000) 2019-12-08T20:20:52.711 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:52.705+0000 1694f700 -1 osd.7 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:34.173162+0000 front 2019-12-08T20:19:34.231535+0000 (oldest deadline 2019-12-08T20:20:18.806907+0000) 2019-12-08T20:20:52.712 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:52.706+0000 1694f700 -1 osd.7 242 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:52.792 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:20:52.785+0000 1694f700 -1 osd.5 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.713584+0000 front 2019-12-08T20:19:36.723267+0000 (oldest deadline 2019-12-08T20:20:19.969000+0000) 2019-12-08T20:20:52.992 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:20:52.989+0000 1694f700 -1 osd.2 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:35.742874+0000 front 2019-12-08T20:19:35.633678+0000 (oldest deadline 2019-12-08T20:20:19.838425+0000) 2019-12-08T20:20:52.993 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:20:52.986+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e242 v242) 2019-12-08T20:20:53.022 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:20:53.018+0000 1694f700 -1 osd.3 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:33.810263+0000 front 2019-12-08T20:19:33.808362+0000 (oldest deadline 2019-12-08T20:20:18.438386+0000) 2019-12-08T20:20:53.256 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:20:53.249+0000 1694f700 -1 osd.4 242 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:19:36.405738+0000 front 2019-12-08T20:19:36.162885+0000 (oldest deadline 2019-12-08T20:20:21.448277+0000) 2019-12-08T20:20:53.822 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:53.816+0000 1694f700 -1 osd.7 243 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:20:54.937 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:20:54.883+0000 1694f700 -1 osd.7 244 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111317 2.1ds0 2:bb286d87:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%eb%8c%8d%e9%90%8b%e0%a4%82%ed%88%8f%e3%9c%86%eb%8c%8b%ed%88%8e%e0%a4%82%ef%84%90%ed%88%8d%e3%9c%84%e5%98%88%ed%88%8e%e8%94%89%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~21532 in=21532b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e242) 2019-12-08T20:21:33.280 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:33.249+0000 33f8b700 -1 osd.4 246 build_incremental_map_msg missing incremental map 246 2019-12-08T20:21:33.434 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:33.382+0000 32f89700 -1 osd.4 246 build_incremental_map_msg missing incremental map 246 2019-12-08T20:21:33.668 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:33.587+0000 38794700 -1 osd.4 246 build_incremental_map_msg missing incremental map 246 2019-12-08T20:21:33.680 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:33.622+0000 38f95700 -1 osd.4 246 build_incremental_map_msg missing incremental map 246 2019-12-08T20:21:34.497 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:34.481+0000 3478c700 -1 osd.4 247 build_incremental_map_msg missing incremental map 246 2019-12-08T20:21:45.444 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:45.438+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:46.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:46.443+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:47.426 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:47.421+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:48.428 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:48.422+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:49.433 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:49.427+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:49.965 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:49.953+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:50.053 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:50.045+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:50.481 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:50.466+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:50.482 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:50.468+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:50.964 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:50.960+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:51.097 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:51.087+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:51.490 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:51.482+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:51.497 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:51.489+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:51.497 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:51.491+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:51.941 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:51.931+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:52.128 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:52.122+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:52.506 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:52.497+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:52.506 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:52.500+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:52.533 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:52.525+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:52.922 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:52.919+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:53.129 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:53.121+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:53.472 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:53.465+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:53.475 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:53.468+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:53.531 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:53.523+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:53.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:53.961+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:21:53.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:53.964+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:54.181 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:54.162+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:54.493 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:54.486+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:54.496 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:54.491+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:54.527 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:54.520+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:54.605 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:21:54.595+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:21:54.959 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:54.954+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:21:54.961 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:54.957+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:55.185 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:55.175+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:55.493 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:55.486+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:55.494 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:55.489+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:55.496 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:55.489+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:55.589 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:21:55.575+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:21:55.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:55.971+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:21:55.979 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:55.975+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:56.212 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:56.204+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:56.531 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:56.521+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:56.533 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:56.525+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:56.549 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:56.542+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:56.594 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:21:56.591+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:21:56.972 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:56.968+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:21:56.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:56.971+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:57.193 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:57.186+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:57.493 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:57.486+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:57.494 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:57.488+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:57.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:57.545+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:57.626 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:21:57.622+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:21:57.985 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:57.980+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:21:57.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:57.985+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:58.191 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:58.184+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:58.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:58.441+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:58.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:58.444+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:58.515 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:58.508+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:58.603 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:21:58.599+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:21:59.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:59.028+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:21:59.033 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:59.030+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:59.230 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:21:59.220+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:21:59.460 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:59.454+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:21:59.471 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:21:59.465+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:21:59.472 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:21:59.465+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:21:59.564 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:21:59.559+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:00.005 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:21:59.999+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:00.006 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:00.003+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:00.257 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:00.250+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:00.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:00.449+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:00.464 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:00.457+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:00.465 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:00.459+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:00.600 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:00.595+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:01.060 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:01.053+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:01.061 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:01.057+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:01.291 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:01.283+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:01.424 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:01.417+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:01.427 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:01.419+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:01.427 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:01.422+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:01.638 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:01.633+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:02.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:02.073+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:02.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:02.074+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:02.332 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:02.325+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:02.445 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:02.437+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:02.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:02.445+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:02.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:02.448+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:02.608 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:02.603+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:03.089 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:03.083+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:03.090 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:03.084+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:03.300 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:03.292+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:03.430 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:03.423+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:03.447 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:03.438+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:03.448 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:03.442+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:03.597 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:03.592+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:03.662 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:03.656+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:04.099 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:04.095+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:04.100 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:04.096+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:04.329 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:04.321+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:04.403 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:04.396+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:04.454 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:04.448+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:04.457 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:04.452+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:04.556 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:04.552+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:04.701 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:04.696+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:05.084 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:05.080+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:05.087 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:05.084+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:05.373 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:05.368+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:05.380 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:05.374+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:05.503 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:05.497+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:05.507 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:05.502+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:05.585 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:05.580+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:05.668 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:05.664+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:06.043 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:06.039+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:06.046 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:06.043+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:06.358 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:06.351+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:06.403 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:06.397+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:06.524 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:06.517+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:06.525 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:06.520+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:06.575 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:06.568+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:06.587 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:06.581+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:06.710 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:06.705+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:07.061 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:07.055+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:07.062 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:07.059+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:07.355 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:07.349+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:07.387 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:07.381+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:07.527 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:07.520+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:07.527 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:07.523+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:07.613 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:07.608+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:07.640 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:07.635+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:07.764 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:07.759+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:08.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:08.098+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:08.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:08.101+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:08.384 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:08.371+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:08.437 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:08.429+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:08.525 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:08.515+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:08.525 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:08.518+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:08.592 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:08.587+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:08.630 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:08.626+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:08.791 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:08.787+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:09.115 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:09.110+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:09.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:09.113+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:09.376 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:09.370+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:09.425 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:09.414+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:09.559 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:09.546+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:09.577 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:09.567+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:09.577 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:09.572+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:09.616 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:09.612+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:09.798 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:09.793+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:10.123 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:10.117+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:10.126 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:10.120+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:10.390 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:10.383+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:10.428 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:10.422+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:10.553 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:10.548+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:10.575 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:10.569+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:10.582 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:10.576+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:10.585 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:10.581+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:10.803 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:10.798+0000 1694f700 -1 osd.6 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:21:30.121377+0000 (oldest deadline 2019-12-08T20:22:10.121377+0000) 2019-12-08T20:22:10.812 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:10.803+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:11.134 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:11.130+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:11.138 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:11.134+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:11.414 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:11.408+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:11.459 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:11.453+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:11.546 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:11.541+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:11.600 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:11.593+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:11.600 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:11.596+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:11.604 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:11.600+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:11.829 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:11.810+0000 1694f700 -1 osd.6 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:21:30.121377+0000 (oldest deadline 2019-12-08T20:22:10.121377+0000) 2019-12-08T20:22:11.830 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:11.814+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:12.092 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:12.085+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:12.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:12.088+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:12.416 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:12.410+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:12.486 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:22:12.480+0000 1694f700 -1 osd.5 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:08.332627+0000 front 2019-12-08T20:21:08.322886+0000 (oldest deadline 2019-12-08T20:21:49.985962+0000) 2019-12-08T20:22:12.562 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:22:12.557+0000 1694f700 -1 osd.2 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.115122+0000 front 2019-12-08T20:21:09.115412+0000 (oldest deadline 2019-12-08T20:21:54.378910+0000) 2019-12-08T20:22:12.635 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:22:12.631+0000 1694f700 -1 osd.3 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:09.313023+0000 front 2019-12-08T20:21:09.273132+0000 (oldest deadline 2019-12-08T20:22:05.993202+0000) 2019-12-08T20:22:12.637 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:12.630+0000 1694f700 -1 osd.7 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.294783+0000 front 2019-12-08T20:21:07.273814+0000 (oldest deadline 2019-12-08T20:21:49.906619+0000) 2019-12-08T20:22:12.637 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:12.633+0000 1694f700 -1 osd.7 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:12.839 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:12.832+0000 1694f700 -1 osd.6 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:21:30.121377+0000 (oldest deadline 2019-12-08T20:22:10.121377+0000) 2019-12-08T20:22:12.844 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:12.834+0000 1694f700 -1 osd.6 248 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:13.067 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:13.062+0000 1694f700 -1 osd.0 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:07.313551+0000 front 2019-12-08T20:21:07.361547+0000 (oldest deadline 2019-12-08T20:21:53.200803+0000) 2019-12-08T20:22:13.069 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:13.066+0000 1694f700 -1 osd.0 248 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:13.373 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:22:13.368+0000 1694f700 -1 osd.4 248 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:21:06.139326+0000 front 2019-12-08T20:21:06.426536+0000 (oldest deadline 2019-12-08T20:21:51.272386+0000) 2019-12-08T20:22:13.818 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:13.780+0000 1694f700 -1 osd.7 249 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:13.971 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:22:13.861+0000 1694f700 -1 osd.6 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e246) 2019-12-08T20:22:14.241 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:22:14.224+0000 1694f700 -1 osd.0 249 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:111500 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5395569424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:14.803 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:22:14.760+0000 1694f700 -1 osd.7 250 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111498 2.5s0 2:a1402f74:::9161132f-da48-40b4-a853-f341a1971d48.4507.614_%ce%a9%ef%84%8f%e8%94%8b%e2%98%83%ef%84%8f%e9%90%89%ed%88%8e%e5%98%86%eb%8c%8b%eb%8c%8e%eb%8c%8e%e5%98%85%e0%a4%82%e5%98%86%eb%8c%8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2823 in=2823b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e244) 2019-12-08T20:22:15.437 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:22:15.434+0000 20e25700 -1 RGWWatcher::handle_error cookie 5395569424 err (107) Transport endpoint is not connected 2019-12-08T20:23:49.893 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:49.886+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:50.913 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:50.908+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:51.870 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:51.866+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:52.861 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:52.857+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:53.852 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:53.848+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:54.852 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:54.848+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:55.830 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:55.825+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:56.800 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:56.796+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:57.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:57.787+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:58.769 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:58.764+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:23:59.809 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:23:59.803+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:00.816 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:00.811+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:01.852 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:01.844+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:02.879 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:02.874+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:03.894 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:03.889+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:03.896 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:03.891+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:03.913 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:03.908+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:04.422 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:04.418+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:04.891 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:04.885+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:04.891 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:04.886+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:04.922 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:04.916+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:05.082 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:05.076+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:05.089 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:05.084+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:05.421 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:05.416+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:05.896 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:05.891+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:05.897 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:05.891+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:05.922 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:05.916+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:06.120 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:06.114+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:06.135 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:06.130+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:06.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:06.204+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:06.400 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:06.395+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:06.854 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:06.849+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:06.857 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:06.852+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:06.970 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:06.965+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:07.096 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:07.090+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:07.164 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:07.159+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:07.181 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:07.176+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:07.413 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:07.407+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:07.839 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:07.834+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:07.840 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:07.836+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:07.985 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:07.978+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:08.054 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:08.048+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:08.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:08.132+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:08.166 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:08.161+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:08.401 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:08.390+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:08.415 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:08.409+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:08.858 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:08.852+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:08.860 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:08.856+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:08.984 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:08.979+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:09.100 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:09.095+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:09.103 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:09.098+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:09.154 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:09.147+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:09.421 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:09.417+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:09.441 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:09.435+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:09.890 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:09.883+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:09.891 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:09.885+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:10.038 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:10.032+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:10.066 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:10.060+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:10.101 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:10.096+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:10.195 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:10.190+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:10.449 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:10.443+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:10.486 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:10.481+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:10.872 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:10.865+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:10.872 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:10.867+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:11.070 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:11.063+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:11.075 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:11.069+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:11.091 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:11.087+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:11.194 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:11.188+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:11.436 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:11.432+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:11.536 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:11.530+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:11.875 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:11.869+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:11.877 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:11.873+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:12.061 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:12.057+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:12.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:12.069+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:12.097 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:12.090+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:12.169 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:12.162+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:12.417 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:12.413+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:12.569 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:12.564+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:12.846 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:12.841+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:12.847 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:12.841+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:13.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:13.073+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:13.078 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:13.074+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:13.098 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:13.092+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:13.130 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:13.124+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:13.436 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:13.431+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:13.592 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:13.587+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:13.813 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:13.807+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:13.813 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:13.808+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:14.048 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:14.043+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:14.089 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:14.082+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:14.123 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:14.119+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:14.131 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:14.125+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:14.425 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:14.420+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:14.643 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:14.639+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:14.825 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:14.818+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:14.826 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:14.820+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:15.029 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:15.024+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:15.113 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:15.107+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:15.122 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:15.117+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:15.175 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:15.170+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:15.478 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:15.473+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:15.613 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:15.608+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:15.875 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:15.869+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:15.876 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:15.870+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:16.051 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:16.046+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:16.123 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:16.117+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:16.136 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:16.129+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:16.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:16.176+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:16.523 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:16.518+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:16.645 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:16.641+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:16.876 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:16.869+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:16.876 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:16.871+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:17.066 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:17.061+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:17.104 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:17.098+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:17.142 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:17.136+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:17.215 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:17.209+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:17.539 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:17.534+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:17.616 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:17.612+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:17.848 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:17.842+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:17.848 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:17.843+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:18.052 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:18.049+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:18.082 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:18.075+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:18.188 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:18.182+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:18.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:18.224+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:18.541 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:18.538+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:18.570 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:18.566+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:18.880 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:18.873+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:18.882 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:18.874+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:19.086 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:19.081+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:19.100 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:19.094+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:19.143 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:19.137+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:19.235 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:19.231+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:19.567 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:19.562+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:19.588 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:19.584+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:19.842 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:19.835+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:19.842 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:19.836+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:20.120 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:20.116+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:20.144 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:20.138+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:20.151 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:20.145+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:20.286 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:20.282+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:20.532 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:20.529+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:20.639 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:20.636+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:20.798 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:20.792+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:20.801 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:20.796+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:21.129 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:21.123+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:21.168 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:21.164+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:21.180 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:21.175+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:21.319 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:21.314+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:21.499 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:21.495+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:21.597 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:21.593+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:21.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:21.760+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:21.767 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:21.761+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:22.097 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:22.090+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:22.147 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:22.143+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:22.154 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:22.148+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:22.302 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:22.297+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:22.523 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:22.518+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:22.619 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:22.615+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:22.746 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:22.735+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:22.748 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:22.741+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:23.145 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:23.140+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:23.184 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:23.178+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:23.186 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:23.182+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:23.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:23.346+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:23.537 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:23.533+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:23.584 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:23.581+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:23.742 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:23.736+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:23.742 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:23.737+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:24.170 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:24.165+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:24.187 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:24.184+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:24.224 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:24.219+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:24.367 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:24.362+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:24.550 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:24.545+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:24.626 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:24.622+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:24.763 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:24.757+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:24.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:24.761+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:25.164 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:25.158+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:25.197 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:25.192+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:25.221 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:25.218+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:25.382 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:25.378+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:25.513 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:25.510+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:25.626 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:25.622+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:25.738 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:25.732+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:25.739 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:25.733+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:26.201 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:26.196+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:26.242 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:26.239+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:26.243 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:26.238+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:26.365 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:26.360+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:26.521 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:26.516+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:26.646 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:26.643+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:26.719 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:26.713+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:26.720 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:26.714+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:27.216 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:27.210+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:27.244 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:27.238+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:27.289 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:27.285+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:27.384 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:27.380+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:27.557 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:27.554+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:27.683 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:27.679+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:27.696 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:27.690+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:27.697 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:27.691+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:28.225 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:28.219+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:28.249 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:28.243+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:28.311 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:28.307+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:28.347 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:28.342+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:28.598 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:28.594+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:28.652 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:28.645+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:28.652 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:28.647+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:28.733 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:28.730+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:29.065 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:41:00.493 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.070 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:41:00.498 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.079 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:41:00.505 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.083 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:41:00.510 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.084 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:41:00.677 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.090 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:41:00.683 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.245 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:29.240+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:29.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:29.264+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:29.302 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:29.298+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:29.312 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:41:00.739 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.317 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:41:00.743 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.360 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:41:00.953 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:29.379 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:29.375+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:29.643 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:29.640+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:29.672 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:29.666+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:29.673 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:29.668+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:29.721 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:29.718+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:30.224 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:30.220+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:30.278 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:30.269+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:30.314 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:30.309+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:30.395 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:30.391+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:30.628 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:30.625+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:30.665 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:30.660+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:30.666 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:30.662+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:30.762 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:30.758+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:31.208 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:31.204+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:31.263 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:31.257+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:31.344 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:31.338+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:31.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:31.406+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:31.632 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:31.628+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:31.665 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:31.659+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:31.666 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:31.660+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:31.805 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:31.802+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:32.217 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:32.214+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:32.293 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:32.287+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:32.392 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:32.385+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:32.418 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:32.414+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:32.633 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:32.627+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:32.634 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:32.628+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:32.647 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:32.644+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:32.787 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:32.784+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:33.262 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:33.258+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:33.320 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:33.312+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:33.364 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:33.356+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:33.401 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:33.391+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:33.630 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:33.625+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:33.637 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:33.631+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:33.638 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:33.633+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:33.743 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:33.739+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:34.285 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:34.280+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:34.326 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:34.319+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:34.361 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:34.355+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:34.446 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:34.442+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:34.605 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:34.601+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:34.638 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:34.633+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:34.639 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:34.633+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:34.700 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:34.696+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:35.267 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:35.263+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:35.320 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:35.315+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:35.403 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:35.398+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:35.477 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:35.473+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:35.606 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:35.600+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:35.607 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:35.601+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:35.655 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:35.650+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:35.704 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:35.699+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:36.253 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:36.248+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:36.375 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:36.365+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:36.440 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:36.433+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:36.499 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:36.494+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:36.635 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:36.628+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:36.636 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:36.630+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:36.641 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:36.637+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:36.693 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:36.688+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:37.289 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:37.285+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:37.420 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:37.416+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:37.453 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:37.448+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:37.537 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:37.530+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:37.622 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:37.617+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:37.672 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:37.667+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:37.673 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:37.668+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:37.706 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:37.702+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:38.267 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:38.262+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:38.412 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:38.407+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:38.461 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:38.455+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:38.493 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:38.488+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:38.641 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:38.637+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:38.716 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:38.711+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:38.717 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:38.712+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:38.747 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:24:38.742+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 45sec e254 v254) 2019-12-08T20:24:38.756 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:38.753+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:39.221 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:39.218+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:39.437 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:39.430+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:39.450 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:39.445+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:39.543 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:39.539+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:39.615 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:39.612+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:39.689 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:39.685+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:39.690 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:39.685+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:39.712 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:39.708+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:40.224 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:40.220+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:40.439 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:40.435+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:40.474 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:40.467+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:40.509 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:40.505+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:40.576 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:40.572+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:40.708 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:40.703+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:40.711 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:40.708+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:40.728 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:40.724+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:41.216 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:41.212+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:41.485 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:41.475+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:41.501 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:41.496+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:41.549 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:41.545+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:41.585 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:41.580+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:41.746 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:41.741+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:41.747 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:41.742+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:41.758 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:41.754+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:42.258 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:42.254+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:42.476 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:42.471+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:42.483 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:42.478+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:42.525 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:42.520+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:42.545 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:42.541+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:42.708 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:42.704+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:42.709 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:42.705+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:42.742 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:42.739+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:43.264 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:24:43.260+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.444974+0000 front 2019-12-08T20:23:21.893006+0000 (oldest deadline 2019-12-08T20:24:05.076708+0000) 2019-12-08T20:24:43.452 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:24:43.446+0000 1694f700 -1 osd.4 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:20.202565+0000 front 2019-12-08T20:23:19.566581+0000 (oldest deadline 2019-12-08T20:24:03.423041+0000) 2019-12-08T20:24:43.500 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:24:43.495+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.833503+0000 front 2019-12-08T20:23:21.831293+0000 (oldest deadline 2019-12-08T20:24:03.654067+0000) 2019-12-08T20:24:43.528 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:24:43.524+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.486699+0000 front 2019-12-08T20:23:21.404343+0000 (oldest deadline 2019-12-08T20:24:04.256155+0000) 2019-12-08T20:24:43.569 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:24:43.565+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:21.832468+0000 front 2019-12-08T20:23:20.442055+0000 (oldest deadline 2019-12-08T20:24:05.476105+0000) 2019-12-08T20:24:43.723 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:43.718+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:18.224136+0000 front 2019-12-08T20:23:18.254041+0000 (oldest deadline 2019-12-08T20:24:03.286438+0000) 2019-12-08T20:24:43.725 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:43.722+0000 1694f700 -1 osd.7 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:43.766 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:24:43.762+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:23:22.884363+0000 front 2019-12-08T20:23:22.749204+0000 (oldest deadline 2019-12-08T20:24:08.008438+0000) 2019-12-08T20:24:43.837 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:24:43.831+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 45sec e254 v254) 2019-12-08T20:24:44.761 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:44.754+0000 1694f700 -1 osd.7 255 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:45.913 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:24:45.903+0000 1694f700 -1 osd.7 256 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:111794 2.1ds0 2:baade4d7:::9161132f-da48-40b4-a853-f341a1971d48.4507.615_%e2%98%83%eb%8c%8c%ef%84%92%e1%a0%84%e9%90%8b%eb%8c%8d%e9%90%8c%e1%a0%82%ef%84%8f%e5%98%87%e3%9c%84%e1%a0%82%e3%9c%84%e8%94%89%e1%a0%81:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~25317 in=25317b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e254) 2019-12-08T20:24:54.019 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:41:25.447 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.034 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:41:25.462 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.058 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:41:25.651 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.061 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:41:25.653 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.081 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:41:25.509 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.081 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:41:25.508 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.089 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:41:25.515 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.141 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:41:25.567 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:24:54.142 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:41:25.735 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:25:17.372 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:17.367+0000 38794700 -1 osd.6 259 build_incremental_map_msg missing incremental map 225 2019-12-08T20:25:17.481 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:17.477+0000 38794700 -1 osd.6 259 build_incremental_map_msg missing incremental map 235 2019-12-08T20:25:42.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:42.193+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:42.234 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:42.227+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:43.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:43.208+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:43.265 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:43.259+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:44.168 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:44.164+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:44.179 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:44.173+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:44.268 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:44.263+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:44.643 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:44.636+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:45.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:45.032+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:45.159 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:45.151+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:45.188 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:45.184+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:45.256 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:45.251+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:45.325 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:45.315+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:45.629 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:45.620+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:46.024 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:46.020+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:46.120 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:46.112+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:46.156 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:46.151+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:46.232 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:46.216+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:46.329 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:46.324+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:46.654 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:46.648+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:47.068 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:47.065+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:47.137 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:47.131+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:47.167 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:47.163+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:47.272 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:47.267+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:47.371 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:47.365+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:47.650 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:47.643+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:48.038 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:48.034+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:48.129 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:48.122+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:48.133 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:48.129+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:48.308 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:48.302+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:48.362 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:48.354+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:48.705 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:48.698+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:49.087 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:49.082+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:49.138 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:49.133+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:49.156 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:49.149+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:49.351 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:49.344+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:49.412 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:49.405+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:49.743 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:49.736+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:50.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:50.072+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:50.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:50.074+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:50.133 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:50.129+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:50.210 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:50.203+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:50.370 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:50.364+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:50.372 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:50.364+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:50.747 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:50.739+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:51.126 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:51.122+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:51.127 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:51.123+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:51.169 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:51.165+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:51.170 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:51.162+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:51.416 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:51.409+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:51.417 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:51.411+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:51.425 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:51.418+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:51.720 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:51.713+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:52.143 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:52.140+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:52.144 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:52.141+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:52.145 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:52.138+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:52.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:52.145+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:52.384 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:52.377+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:52.407 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:52.396+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:52.408 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:52.398+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:52.688 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:52.681+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:53.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:53.111+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:53.119 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:53.116+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:53.119 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:53.114+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:53.190 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:53.184+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:53.397 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:53.391+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:53.414 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:53.406+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:53.416 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:53.409+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:53.686 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:53.679+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:54.128 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:54.123+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:54.144 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:54.137+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:54.146 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:54.139+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:54.222 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:54.216+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:54.376 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:54.368+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:54.460 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:54.454+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:54.464 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:54.459+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:54.704 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:54.696+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:55.097 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:55.094+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:55.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:55.159+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:55.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:55.161+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:55.202 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:55.194+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:55.373 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:55.367+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:55.452 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:55.444+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:55.453 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:55.447+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:55.719 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:55.712+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:56.091 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:56.087+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:56.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:56.191+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:56.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:56.194+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:56.209 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:56.202+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:56.360 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:56.344+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:56.489 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:56.482+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:56.489 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:56.484+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:56.759 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:56.750+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:57.055 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:57.051+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:57.179 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:57.174+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:57.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:57.176+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:57.261 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:57.254+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:57.355 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:57.348+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:57.529 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:57.521+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:57.529 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:57.524+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:57.725 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:57.717+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:58.078 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:58.074+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:58.178 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:58.173+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:58.179 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:58.174+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:58.281 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:58.275+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:58.368 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:58.360+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:58.510 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:58.502+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:58.511 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:58.505+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:58.783 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:58.774+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:25:59.097 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:25:59.092+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:25:59.159 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:59.155+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:25:59.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:25:59.157+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:25:59.284 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:25:59.275+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:25:59.422 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:25:59.416+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:25:59.477 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:59.470+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:25:59.477 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:25:59.472+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:25:59.777 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:25:59.770+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:00.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:00.115+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:00.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:00.118+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:00.145 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:00.140+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:00.317 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:00.311+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:00.452 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:00.444+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:00.517 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:00.509+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:00.519 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:00.512+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:00.844 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:00.835+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:01.143 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:01.139+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:01.158 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:01.152+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:01.162 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:01.159+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:01.363 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:01.357+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:01.427 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:01.421+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:01.491 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:01.486+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:01.493 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:01.489+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:01.821 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:01.813+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:02.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:02.135+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:02.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:02.138+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:02.190 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:02.186+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:02.389 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:02.380+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:02.408 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:02.401+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:02.489 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:02.481+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:02.491 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:02.484+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:02.819 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:02.810+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:03.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:03.182+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:03.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:03.186+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:03.211 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:03.207+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:03.410 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:03.403+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:03.421 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:03.414+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:03.540 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:03.532+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:03.540 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:03.535+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:03.772 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:03.765+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:04.199 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:04.194+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:04.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:04.196+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:04.234 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:04.230+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:04.386 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:04.379+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:04.388 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:04.381+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:04.536 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:04.530+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:04.539 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:04.535+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:04.789 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:04.782+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:05.189 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:05.183+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:05.192 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:05.186+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:05.245 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:05.241+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:05.350 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:05.343+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:05.437 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:05.427+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:05.534 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:05.527+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:05.535 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:05.529+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:05.799 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:05.791+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:06.239 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:06.232+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:06.239 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:06.236+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:06.267 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:06.264+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:06.315 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:06.305+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:06.434 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:06.428+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:06.537 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:06.530+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:06.539 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:06.533+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:06.760 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:06.754+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:07.221 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:07.218+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:07.269 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:07.264+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:07.269 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:07.266+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:07.272 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:07.265+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:07.398 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:07.391+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:07.514 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:07.505+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:07.515 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:07.509+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:07.803 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:07.796+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:08.178 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:08.172+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:08.310 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:08.302+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:08.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:08.305+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:08.314 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:08.308+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:08.423 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:08.417+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:08.543 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:08.535+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:08.544 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:08.538+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:08.697 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:08.691+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:08.805 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:08.797+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:09.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:09.219+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:09.273 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:09.269+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:09.274 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:09.270+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:09.293 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:09.286+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:09.412 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:09.401+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:09.595 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:09.586+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:09.596 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:09.590+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:09.736 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:09.728+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:09.847 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:09.841+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:10.204 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:10.199+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:10.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:10.250+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:10.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:10.253+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:10.275 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:10.269+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:10.437 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:10.429+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:10.628 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:10.620+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:10.629 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:10.623+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:10.686 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:10.682+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:10.861 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:10.854+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:11.208 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:11.203+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:11.226 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:11.221+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:11.226 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:11.223+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:11.273 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:11.265+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:11.406 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:11.400+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:11.655 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:11.647+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:11.656 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:11.650+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:11.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:11.667+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:11.880 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:11.871+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:12.173 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:12.169+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:12.236 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:12.231+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:12.237 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:12.233+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:12.276 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:12.269+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:12.420 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:12.413+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:12.654 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:12.647+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:12.657 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:12.652+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:12.726 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:12.722+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:12.922 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:12.915+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:13.160 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:13.155+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:13.199 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:13.194+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:13.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:13.199+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:13.243 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:13.236+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:13.386 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:13.378+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:13.695 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:13.687+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:13.696 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:13.691+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:13.765 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:13.761+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:13.934 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:13.924+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:14.129 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:14.124+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:14.221 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:14.214+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:14.249 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:14.244+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:14.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:14.247+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:14.412 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:14.403+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:14.735 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:14.726+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:14.736 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:14.729+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:14.801 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:14.796+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:14.939 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:14.933+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:15.117 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:15.113+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:15.275 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:15.268+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:15.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:15.289+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:15.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:15.292+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:15.448 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:15.441+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:15.755 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:15.749+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:15.756 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:15.750+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:15.772 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:15.768+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:15.905 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:15.898+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:16.083 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:16.079+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:16.253 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:16.247+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:16.263 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:16.259+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:16.266 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:16.263+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:16.449 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:16.442+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:16.728 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:16.722+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:16.729 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:16.721+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:16.733 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:16.727+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:16.910 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:16.902+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:17.046 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:17.042+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:17.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:17.245+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:17.252 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:17.247+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:17.268 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:17.262+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:17.500 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:17.489+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:17.694 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:17.686+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:17.719 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:17.711+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:17.720 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:17.714+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:17.945 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:17.937+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:18.026 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:18.019+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:18.222 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:18.212+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:18.222 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:18.214+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:18.236 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:18.229+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:18.452 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:18.445+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:18.682 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:18.674+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:18.733 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:18.727+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:18.734 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:18.729+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:18.909 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:18.904+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:19.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:19.067+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:19.188 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:26:19.183+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e259 v259) 2019-12-08T20:26:19.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:19.187+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:19.198 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:19.191+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:19.237 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:19.231+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:19.435 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:19.428+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:19.667 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:19.662+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:19.759 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:19.752+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:19.759 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:19.754+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:19.869 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:19.864+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:20.034 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:20.028+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:20.197 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:20.192+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:20.242 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:20.236+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:20.243 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:20.240+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:20.410 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:20.403+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:20.665 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:20.658+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:20.718 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:20.709+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:20.719 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:20.714+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:20.915 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:20.910+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:21.043 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:21.037+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:21.176 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:21.169+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:21.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:21.260+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:21.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:21.263+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:21.414 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:21.407+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:21.706 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:21.700+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:21.735 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:21.729+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:21.736 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:21.732+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:21.879 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:21.873+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:22.066 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:22.060+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:22.219 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:22.214+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:22.223 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:22.215+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:22.225 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:22.219+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:22.371 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:22.364+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:22.715 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:22.709+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:22.717 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:22.712+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:22.759 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:22.754+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:22.905 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:22.900+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:23.029 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:23.015+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:23.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:23.232+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:23.242 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:23.237+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:23.257 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:23.250+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:23.424 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:23.418+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:23.719 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:23.712+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:23.720 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:23.715+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:23.809 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:23.804+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:23.926 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:23.920+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:24.027 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:24.022+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:24.206 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:26:24.199+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e259 v259) 2019-12-08T20:26:24.245 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:24.239+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:24.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:24.250+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:24.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:24.252+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:24.413 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:24.406+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:24.757 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:24.751+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:24.759 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:24.754+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:24.773 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:24.766+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:24.919 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:24.908+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:25.014 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:25.009+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:25.204 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:25.198+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:25.216 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:25.209+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:25.217 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:25.212+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:25.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:25.449+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:25.762 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:25.758+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:25.764 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:25.759+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:25.768 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:25.764+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:25.951 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:25.945+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:26.042 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:26.037+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:26.209 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:26.203+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:26.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:26.221+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:26.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:26.225+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:26.455 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:26.449+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:26.806 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:26.801+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:26.808 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:26.803+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:26.812 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:26.808+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:26.959 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:26.954+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:27.038 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:27.032+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:27.176 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:27.170+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:27.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:27.247+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:27.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:27.251+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:27.492 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:27.486+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:27.784 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:27.779+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:27.809 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:27.802+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:27.809 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:27.805+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:28.004 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:27.998+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:28.007 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:28.001+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:28.137 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:28.131+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:28.274 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:28.260+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:28.274 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:28.262+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:28.537 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:26:28.531+0000 1694f700 -1 osd.4 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.928758+0000 front 2019-12-08T20:25:00.032666+0000 (oldest deadline 2019-12-08T20:25:43.731851+0000) 2019-12-08T20:26:28.809 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:28.804+0000 1694f700 -1 osd.6 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:25:10.855369+0000 (oldest deadline 2019-12-08T20:25:50.855369+0000) 2019-12-08T20:26:28.813 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:26:28.809+0000 1694f700 -1 osd.6 259 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e258) 2019-12-08T20:26:28.831 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:26:28.827+0000 1694f700 -1 osd.2 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.573760+0000 front 2019-12-08T20:24:56.583083+0000 (oldest deadline 2019-12-08T20:26:08.374639+0000) 2019-12-08T20:26:28.980 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:26:28.975+0000 1694f700 -1 osd.7 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:25:00.166485+0000 front 2019-12-08T20:25:00.112393+0000 (oldest deadline 2019-12-08T20:25:44.200812+0000) 2019-12-08T20:26:29.018 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:26:29.015+0000 1694f700 -1 osd.3 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:56.758424+0000 front 2019-12-08T20:24:56.637266+0000 (oldest deadline 2019-12-08T20:25:41.191513+0000) 2019-12-08T20:26:29.107 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:26:29.101+0000 1694f700 -1 osd.5 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:58.751386+0000 front 2019-12-08T20:24:58.767616+0000 (oldest deadline 2019-12-08T20:25:42.226713+0000) 2019-12-08T20:26:29.243 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:26:29.238+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e259 v259) 2019-12-08T20:26:29.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:29.287+0000 1694f700 -1 osd.0 259 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:24:59.448419+0000 front 2019-12-08T20:24:59.517050+0000 (oldest deadline 2019-12-08T20:25:44.139785+0000) 2019-12-08T20:26:29.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:29.290+0000 1694f700 -1 osd.0 259 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:30.331 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:30.327+0000 1694f700 -1 osd.0 260 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112025 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:31.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:26:31.294+0000 1694f700 -1 osd.0 261 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112053 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1471573632 gen 1] snapc 0=[] ondisk+write+known_if_redirected e259) 2019-12-08T20:26:31.739 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:26:31.730+0000 20e25700 -1 RGWWatcher::handle_error cookie 1471573632 err (107) Transport endpoint is not connected 2019-12-08T20:26:59.630 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:43:31.058 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.641 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:43:31.068 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.678 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:43:31.104 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.691 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:43:31.117 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.744 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:43:31.336 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.750 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:43:31.343 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.848 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:43:31.274 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.877 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:43:31.305 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:26:59.985 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:43:31.578 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:27:53.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:53.507+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:53.546 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:53.540+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:54.004 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:27:53.999+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:27:54.486 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:54.480+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:54.514 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:54.509+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:54.569 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:27:54.564+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:27:55.025 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:27:55.020+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:27:55.199 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:27:55.193+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:27:55.363 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:27:55.357+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:27:55.473 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:55.467+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:55.476 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:55.471+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:55.588 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:27:55.582+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:27:56.066 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:27:56.061+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:27:56.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:27:56.205+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:27:56.212 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:27:56.205+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:27:56.370 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:27:56.365+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:27:56.451 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:56.446+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:56.491 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:56.486+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:56.625 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:27:56.619+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:27:57.041 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:27:57.036+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:27:57.190 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:27:57.176+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:27:57.191 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:27:57.181+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:27:57.408 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:27:57.403+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:27:57.410 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:57.404+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:57.518 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:57.512+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:57.665 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:27:57.658+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:27:58.001 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:27:57.995+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:27:58.180 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:27:58.175+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:27:58.195 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:27:58.190+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:27:58.396 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:27:58.391+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:27:58.447 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:58.441+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:58.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:58.549+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:58.672 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:27:58.666+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:27:59.011 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:27:59.005+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:27:59.188 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:27:59.183+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:27:59.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:27:59.194+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:27:59.419 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:27:59.415+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:27:59.423 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:27:59.417+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:27:59.581 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:27:59.573+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:27:59.627 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:27:59.621+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:00.045 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:00.040+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:00.195 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:00.191+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:00.200 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:00.195+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:00.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:00.405+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:00.422 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:00.416+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:00.551 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:00.545+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:00.664 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:00.658+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:01.049 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:01.044+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:01.176 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:01.172+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:01.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:01.219+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:01.417 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:01.413+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:01.428 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:01.424+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:01.506 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:01.501+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:01.715 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:01.708+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:02.022 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:02.019+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:02.173 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:02.169+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:02.216 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:02.212+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:02.444 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:02.441+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:02.447 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:02.442+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:02.465 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:02.460+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:02.720 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:02.715+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:03.065 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:03.061+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:03.198 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:03.194+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:03.245 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:03.241+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:03.438 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:03.433+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:03.473 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:03.468+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:03.492 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:03.486+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:03.681 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:03.676+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:04.098 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:04.094+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:04.155 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:04.152+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:04.263 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:04.257+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:04.448 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:04.441+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:04.477 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:04.461+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:04.485 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:04.481+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:04.709 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:04.705+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:05.118 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:05.115+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:05.177 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:05.173+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:05.275 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:05.271+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:05.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:05.441+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:05.470 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:05.463+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:05.481 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:05.477+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:05.723 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:05.716+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:06.169 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:06.165+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:06.208 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:06.204+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:06.241 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:06.237+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:06.451 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:06.446+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:06.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:06.451+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:06.488 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:06.483+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:06.750 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:06.745+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:07.190 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:07.185+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:07.215 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:07.211+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:07.227 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:07.224+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:07.422 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:07.418+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:07.488 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:07.483+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:07.533 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:07.528+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:07.783 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:07.778+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:08.167 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:08.162+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:08.175 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:08.170+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:08.217 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:08.214+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:08.469 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:08.465+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:08.487 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:08.483+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:08.496 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:08.491+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:08.787 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:08.782+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:09.133 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:09.129+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:09.192 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:09.188+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:09.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:09.222+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:09.484 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:09.479+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:09.486 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:09.482+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:09.524 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:09.520+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:09.764 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:09.758+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:10.105 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:10.099+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:10.171 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:10.167+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:10.188 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:10.182+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:10.457 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:10.446+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:10.473 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:10.466+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:10.498 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:10.494+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:10.751 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:10.746+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:11.099 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:11.095+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:11.213 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:11.209+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:11.214 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:11.211+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:11.458 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:11.453+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:11.462 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:11.457+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:11.463 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:11.458+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:11.772 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:11.767+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:12.096 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:12.091+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:12.214 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:12.207+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:12.243 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:12.239+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:12.431 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:12.426+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:12.486 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:12.481+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:12.497 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:12.492+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:12.748 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:12.744+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:13.143 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:13.135+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:13.176 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:13.172+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:13.239 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:13.233+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:13.451 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:13.445+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:13.480 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:13.476+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:13.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:13.534+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:13.785 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:13.778+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:14.173 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:14.169+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:14.175 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:14.171+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:14.224 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:14.219+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:14.427 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:14.423+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:14.479 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:14.473+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:14.591 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:14.583+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:14.793 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:14.787+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:15.160 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:15.157+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:15.182 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:15.179+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:15.205 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:15.201+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:15.463 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:15.456+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:15.473 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:15.469+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:15.629 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:15.624+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:15.826 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:15.821+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:16.204 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:16.198+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:16.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:16.219+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:16.242 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:16.238+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:16.474 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:16.470+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:16.507 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:16.501+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:16.659 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:16.653+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:16.833 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:16.827+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:17.212 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:17.209+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:17.228 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:17.224+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:17.274 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:17.270+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:17.465 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:17.460+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:17.521 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:17.514+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:17.675 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:17.669+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:17.833 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:17.827+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:18.199 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:18.195+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:18.230 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:18.226+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:18.309 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:18.305+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:18.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:18.430+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:18.545 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:18.539+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:18.713 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:18.707+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:18.831 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:18.825+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:19.237 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:19.232+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:19.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:19.264+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:19.277 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:19.274+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:19.446 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:19.442+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:19.568 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:19.563+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:19.766 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:19.759+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:19.825 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:19.819+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:20.207 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:20.203+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:20.244 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:20.240+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:20.298 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:20.293+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:20.454 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:20.449+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:20.597 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:20.590+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:20.720 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:20.714+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:20.791 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:20.785+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:21.122 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:44:52.550 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.131 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:44:52.559 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.153 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:44:52.740 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.153 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:44:52.745 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.156 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:44:52.580 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.161 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:44:52.587 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.212 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:21.208+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:21.237 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:44:52.665 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.260 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:44:52.852 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.265 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:21.262+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:21.268 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:21.264+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:21.273 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:44:52.699 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:28:21.505 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:21.501+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:21.644 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:21.639+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:21.715 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:21.709+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:21.775 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:21.770+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:22.182 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:22.178+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:22.284 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:22.280+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:22.284 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:22.280+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:22.520 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:22.515+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:22.650 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:22.643+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:22.709 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:22.704+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:22.812 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:22.806+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:23.184 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:23.180+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:23.314 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:23.310+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:23.315 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:23.311+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:23.495 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:23.491+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:23.643 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:23.638+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:23.664 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:23.660+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:23.842 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:23.836+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:24.153 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:24.149+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:24.273 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:24.269+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:24.281 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:24.277+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:24.535 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:24.531+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:24.637 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:24.631+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:24.683 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:24.677+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:24.887 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:24.881+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:25.198 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:25.194+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:25.263 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:25.259+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:25.336 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:25.332+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:25.537 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:25.532+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:25.667 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:25.662+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:25.721 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:25.715+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:25.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:25.855+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:26.210 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:26.206+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:26.307 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:26.303+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:26.370 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:26.366+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:26.578 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:26.574+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:26.727 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:26.720+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:26.771 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:26.765+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:26.865 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:26.858+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:27.258 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:27.254+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:27.321 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:27.317+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:27.414 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:27.410+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:27.603 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:27.598+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:27.728 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:27.722+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:27.785 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:27.779+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:27.892 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:27.885+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:28.213 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:28.209+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:28.299 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:28.295+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:28.380 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:28.376+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:28.603 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:28.599+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:28.743 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:28.737+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:28.746 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:28.740+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:28.845 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:28.840+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:29.241 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:28:29.237+0000 1694f700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:10.818347+0000 front 2019-12-08T20:27:16.649940+0000 (oldest deadline 2019-12-08T20:27:53.942078+0000) 2019-12-08T20:28:29.266 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:28:29.262+0000 1694f700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.020061+0000 front 2019-12-08T20:27:10.677999+0000 (oldest deadline 2019-12-08T20:27:54.921547+0000) 2019-12-08T20:28:29.370 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:28:29.359+0000 1694f700 -1 osd.3 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.376527+0000 front 2019-12-08T20:27:14.982102+0000 (oldest deadline 2019-12-08T20:27:55.981217+0000) 2019-12-08T20:28:29.617 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:28:29.612+0000 1694f700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:13.259187+0000 front 2019-12-08T20:27:12.908296+0000 (oldest deadline 2019-12-08T20:27:55.186631+0000) 2019-12-08T20:28:29.717 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:28:29.711+0000 1694f700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:11.457948+0000 front 2019-12-08T20:27:14.986170+0000 (oldest deadline 2019-12-08T20:27:53.191280+0000) 2019-12-08T20:28:29.744 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:28:29.738+0000 1694f700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:14.986012+0000 front 2019-12-08T20:27:11.098374+0000 (oldest deadline 2019-12-08T20:27:53.190453+0000) 2019-12-08T20:28:29.853 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:28:29.846+0000 1694f700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:27:16.825167+0000 front 2019-12-08T20:27:14.646430+0000 (oldest deadline 2019-12-08T20:27:54.558835+0000) 2019-12-08T20:28:29.980 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:28:29.972+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 44sec e264 v264) 2019-12-08T20:29:00.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:00.116+0000 1694f700 -1 osd.7 266 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112488 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 821799840 gen 73] snapc 0=[] ondisk+write+known_if_redirected e265) 2019-12-08T20:29:11.821 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:11.806+0000 3378a700 -1 osd.6 269 build_incremental_map_msg missing incremental map 269 2019-12-08T20:29:11.991 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:11.979+0000 37792700 -1 osd.6 269 build_incremental_map_msg missing incremental map 269 2019-12-08T20:29:15.853 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:15.848+0000 37792700 -1 osd.6 270 build_incremental_map_msg missing incremental map 269 2019-12-08T20:29:21.780 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:21.776+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:22.783 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:22.779+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:23.814 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:23.810+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:24.854 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:24.850+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:25.862 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:25.857+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:26.912 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:26.908+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:27.282 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:27.277+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:27.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:27.888+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:28.321 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:28.316+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:28.593 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:28.587+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:28.881 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:28.876+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:29.291 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:29.285+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:29.588 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:29.582+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:29.836 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:29.833+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:30.109 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:30.103+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:30.292 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:30.287+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:30.590 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:30.583+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:30.615 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:30.611+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:30.818 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:30.813+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:30.820 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:30.817+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:31.086 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:31.080+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:31.260 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:31.254+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:31.624 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:31.620+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:31.630 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:31.623+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:31.864 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:31.858+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:31.866 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:31.863+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:32.094 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:32.087+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:32.244 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:32.241+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:32.449 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:32.442+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:32.650 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:32.645+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:32.668 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:32.660+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:32.822 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:32.815+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:32.823 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:32.819+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:33.146 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:33.135+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:33.234 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:33.229+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:33.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:33.424+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:33.674 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:33.666+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:33.697 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:33.692+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:33.787 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:33.781+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:33.788 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:33.784+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:34.160 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:34.153+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:34.194 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:34.189+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:34.480 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:34.470+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:34.675 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:34.672+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:34.677 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:34.670+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:34.765 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:34.760+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:34.765 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:34.762+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:35.184 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:35.177+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:35.194 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:35.190+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:35.473 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:35.466+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:35.678 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:35.673+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:35.700 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:35.694+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:35.740 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:35.735+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:35.740 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:35.737+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:36.158 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:36.154+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:36.170 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:36.163+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:36.498 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:36.491+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:36.660 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:36.656+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:36.702 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:36.695+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:36.723 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:36.718+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:36.727 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:36.723+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:37.113 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:37.109+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:37.162 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:37.155+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:37.548 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:37.542+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:37.673 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:37.669+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:37.702 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:37.695+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:37.761 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:37.755+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:37.761 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:37.758+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:38.123 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:38.118+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:38.200 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:38.192+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:38.523 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:38.516+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:38.627 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:38.623+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:38.661 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:38.653+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:38.773 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:38.768+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:38.773 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:38.770+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:39.145 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:39.139+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:39.249 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:39.242+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:39.537 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:39.530+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:39.618 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:39.614+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:39.642 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:39.635+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:39.765 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:39.761+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:39.766 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:39.763+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:40.145 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:40.141+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:40.285 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:40.278+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:40.582 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:40.576+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:40.640 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:40.635+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:40.692 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:40.684+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:40.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:40.720+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:40.726 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:40.722+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:41.152 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:41.147+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:41.337 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:41.330+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:41.565 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:41.558+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:41.665 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:41.660+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:41.710 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:41.705+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:41.713 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:41.710+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:41.737 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:41.729+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:42.115 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:42.110+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:42.204 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:42.200+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:42.338 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:42.330+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:42.591 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:42.586+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:42.631 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:42.626+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:42.724 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:42.719+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:42.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:42.721+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:42.741 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:42.735+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:43.140 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:43.137+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:43.256 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:43.251+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:43.367 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:43.361+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:43.598 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:43.588+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:43.607 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:43.604+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:43.693 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:43.688+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:43.694 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:43.690+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:43.721 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:43.715+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:44.172 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:44.169+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:44.306 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:44.302+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:44.365 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:44.360+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:44.641 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:44.635+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:44.645 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:44.640+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:44.665 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:44.661+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:44.667 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:44.664+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:44.731 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:44.724+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:45.154 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:45.149+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:45.314 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:45.311+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:45.354 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:45.348+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:45.656 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:45.651+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:45.681 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:45.675+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:45.682 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:45.677+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:45.684 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:45.679+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:45.722 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:45.716+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:46.137 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:46.134+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:46.366 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:46.360+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:46.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:46.381+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:46.665 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:46.660+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:46.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:46.702+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:46.707 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:46.704+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:46.709 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:46.702+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:46.731 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:46.724+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:47.119 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:47.115+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:47.346 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:47.341+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:47.368 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:47.362+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:47.664 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:47.658+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:47.724 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:47.720+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:47.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:47.721+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:47.755 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:47.749+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:47.765 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:47.757+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:48.138 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:48.132+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:48.139 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:48.134+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:48.366 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:48.362+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:48.384 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:48.376+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:48.674 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:48.668+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:48.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:48.720+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:48.729 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:48.725+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:48.762 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:48.757+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:48.769 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:48.764+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:49.126 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:49.118+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:49.127 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:49.122+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:49.369 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:49.364+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:49.414 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:49.408+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:49.666 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:49.661+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:49.725 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:49.718+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:49.740 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:49.734+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:49.767 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:49.763+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:49.768 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:49.764+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:50.122 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:50.117+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:50.125 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:50.121+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:50.323 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:50.319+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:50.448 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:50.443+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:50.673 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:50.669+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:50.729 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:50.724+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:50.766 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:50.760+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:50.818 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:50.812+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:50.819 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:50.815+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:51.143 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:51.135+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:51.148 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:51.144+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:51.316 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:51.312+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:51.443 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:51.428+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:51.747 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:51.712+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:51.757 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:51.750+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:51.765 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:51.758+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:51.853 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:51.848+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:51.854 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:51.850+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:52.132 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:52.125+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:52.133 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:52.128+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:52.309 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:52.304+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:52.433 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:52.427+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:52.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:52.667+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:52.801 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:52.793+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:52.801 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:52.793+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:52.857 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:52.852+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:52.858 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:52.854+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:53.097 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:53.092+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:53.098 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:53.093+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:53.336 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:53.332+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:53.461 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:53.455+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:53.675 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:53.672+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:53.785 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:53.779+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:53.823 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:53.816+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:53.847 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:53.842+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:53.849 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:53.844+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:54.089 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:54.082+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:54.091 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:54.086+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:54.306 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:54.301+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:54.456 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:54.450+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:54.636 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:54.632+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:54.772 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:54.767+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:54.833 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:54.828+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:54.875 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:54.871+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:54.876 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:54.873+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:55.088 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:55.081+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:55.090 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:55.084+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:55.343 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:55.338+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:55.438 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:55.431+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:55.680 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:55.675+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:55.765 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:55.759+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:55.853 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:55.847+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:55.884 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:55.878+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:55.884 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:55.881+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:56.092 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:56.084+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:56.093 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:56.087+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:56.321 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:56.316+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:56.466 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:56.459+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:56.646 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:56.642+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:56.786 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:56.779+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:56.825 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:56.819+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:56.912 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:56.908+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:56.913 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:56.909+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:57.103 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:57.095+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:57.104 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:57.098+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:57.324 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:57.319+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:57.505 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:57.498+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:57.657 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:57.651+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:57.745 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:57.738+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:57.802 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:57.795+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:57.871 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:57.864+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:57.872 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:57.868+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:58.086 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:58.078+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:58.087 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:58.081+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:58.312 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:58.307+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:58.463 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:58.455+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:58.706 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:58.703+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:58.764 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:58.756+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:58.805 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:58.797+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:58.873 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:58.866+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:58.874 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:58.870+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:29:59.055 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:59.047+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:29:59.056 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:29:59.050+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:29:59.282 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:29:59.278+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:29:59.512 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:29:59.506+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:29:59.708 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:29:59.704+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:29:59.726 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:29:59.720+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:29:59.758 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:29:59.752+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:29:59.894 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:59.889+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:29:59.895 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:29:59.891+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:00.075 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:00.052+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:00.075 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:00.059+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:00.242 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:30:00.237+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e270 v270) 2019-12-08T20:30:00.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:00.298+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:00.528 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:00.521+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:00.683 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:00.676+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:00.724 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:00.720+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:00.808 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:00.799+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:00.876 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:00.871+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:00.879 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:00.875+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:01.026 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:01.018+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:01.030 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:01.023+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:01.333 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:01.328+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:01.525 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:01.519+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:01.732 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:01.726+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:01.765 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:01.760+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:01.827 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:01.820+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:01.888 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:01.884+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:01.891 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:01.887+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:02.080 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:02.058+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:02.109 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:02.068+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:02.298 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:02.294+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:02.551 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:02.544+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:02.696 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:02.689+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:02.728 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:02.723+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:02.807 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:02.800+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:02.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:02.889+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:02.894 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:02.890+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:03.042 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:03.034+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:03.043 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:03.037+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:03.329 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:03.326+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:03.518 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:03.511+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:03.659 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:03.651+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:03.684 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:03.679+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:03.766 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:03.759+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:03.899 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:03.887+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:03.899 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:03.890+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:04.096 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:04.087+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:04.097 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:04.091+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:04.330 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:04.325+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:04.494 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:04.486+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:04.617 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:04.609+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:04.641 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:04.636+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:04.741 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:04.733+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:04.935 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:04.930+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:04.936 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:04.931+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:05.107 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:05.100+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:05.113 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:05.107+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:05.264 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:30:05.258+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e270 v270) 2019-12-08T20:30:05.299 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:05.295+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:05.469 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:05.461+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:05.594 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:05.590+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:05.610 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:05.603+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:05.709 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:05.702+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:05.928 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:05.921+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:05.929 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:05.924+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:06.149 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:06.143+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:06.153 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:06.148+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:06.259 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:06.254+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:06.521 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:06.511+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:06.567 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:06.560+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:06.630 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:06.625+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:06.671 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:06.664+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:06.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:06.949+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:06.958 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:06.955+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:07.174 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:07.166+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:07.175 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:07.169+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:07.300 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:07.295+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:07.546 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:07.539+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:07.604 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:07.598+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:07.648 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:07.641+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:07.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:07.667+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:07.980 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:07.974+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:07.983 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:07.978+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:08.199 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:08.192+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:08.200 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:08.194+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:08.291 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:08.286+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:08.535 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:08.527+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:08.610 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:08.602+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:08.622 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:08.614+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:08.716 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:08.711+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:08.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:08.941+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:08.947 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:08.942+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:09.170 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:09.161+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:09.171 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:09.165+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:09.310 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:09.305+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:09.569 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:09.561+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:09.585 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:09.579+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:09.656 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:09.646+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:09.709 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:09.704+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:09.970 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:09.964+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:09.971 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:09.967+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:10.219 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:10.211+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:10.220 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:10.214+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:10.281 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:30:10.273+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e270 v270) 2019-12-08T20:30:10.290 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:10.285+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:10.527 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:10.520+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:10.602 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:10.585+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:10.613 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:10.606+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:10.673 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:10.669+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:10.952 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:10.947+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:10.956 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:10.950+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:11.233 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:11.226+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:11.234 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:11.228+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:11.255 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:11.251+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:11.541 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:11.533+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:11.590 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:11.583+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:11.610 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:11.603+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:11.690 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:11.683+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:11.999 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:11.993+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:12.000 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:11.995+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:12.217 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:12.209+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:12.218 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:12.212+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:12.305 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:12.300+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:12.550 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:12.544+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:12.553 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:12.546+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:12.665 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:12.658+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:12.670 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:12.663+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:13.021 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:13.016+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:13.023 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:13.019+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:13.235 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:13.228+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:13.237 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:13.231+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:13.304 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:13.297+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:13.563 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:13.556+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:13.600 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:13.592+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:13.657 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:13.650+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:13.675 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:13.668+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:14.010 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:14.001+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:14.011 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:14.005+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:14.217 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:14.209+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:14.219 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:14.212+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:14.337 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:14.332+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:14.590 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:14.583+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:14.639 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:14.632+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:14.639 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:14.632+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:14.710 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:14.703+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:15.048 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:15.042+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:15.053 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:15.049+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:15.205 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:15.191+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:15.205 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:15.193+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:15.297 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:30:15.292+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e270 v270) 2019-12-08T20:30:15.312 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:15.307+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:15.551 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:15.545+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:15.617 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:15.611+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:15.652 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:15.646+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:15.724 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:15.720+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:16.010 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:16.006+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:16.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:16.009+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:16.221 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:16.215+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:16.222 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:16.217+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:16.298 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:16.294+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:16.534 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:16.527+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:16.652 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:16.646+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:16.660 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:16.653+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:16.758 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:16.754+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:17.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:17.027+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:17.034 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:17.029+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:17.224 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:17.217+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:17.228 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:17.223+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:17.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:17.258+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:17.576 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:17.569+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:17.660 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:17.653+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:17.692 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:17.683+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:17.749 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:17.745+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:18.021 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:18.015+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:18.021 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:18.017+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:18.269 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:18.260+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:18.270 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:18.264+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:18.281 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:18.278+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:18.569 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:18.561+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:18.655 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:18.647+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:18.655 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:18.648+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:18.757 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:18.752+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:19.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:19.036+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:19.043 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:19.040+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:19.273 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:19.269+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:19.299 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:19.292+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:19.301 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:19.295+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:19.612 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:30:19.605+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:47.175107+0000 front 2019-12-08T20:28:46.983298+0000 (oldest deadline 2019-12-08T20:29:29.834926+0000) 2019-12-08T20:30:19.622 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:30:19.615+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.770493+0000 front 2019-12-08T20:28:45.828659+0000 (oldest deadline 2019-12-08T20:29:29.104943+0000) 2019-12-08T20:30:19.691 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:30:19.684+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.802785+0000 front 2019-12-08T20:28:46.765855+0000 (oldest deadline 2019-12-08T20:29:32.049151+0000) 2019-12-08T20:30:19.780 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:30:19.775+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:45.992704+0000 front 2019-12-08T20:28:46.080579+0000 (oldest deadline 2019-12-08T20:29:28.248817+0000) 2019-12-08T20:30:20.060 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:20.056+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:46.433283+0000 front 2019-12-08T20:28:46.444526+0000 (oldest deadline 2019-12-08T20:29:29.909155+0000) 2019-12-08T20:30:20.066 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:20.060+0000 1694f700 -1 osd.0 270 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:20.253 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:30:20.249+0000 1694f700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:28:41.765508+0000 front 2019-12-08T20:28:41.775199+0000 (oldest deadline 2019-12-08T20:29:27.166855+0000) 2019-12-08T20:30:20.320 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:20.313+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:29:07.915322+0000 (oldest deadline 2019-12-08T20:29:47.915322+0000) 2019-12-08T20:30:20.324 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:30:20.319+0000 1694f700 -1 osd.6 270 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-12-08T20:30:20.344 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:30:20.340+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e270 v270) 2019-12-08T20:30:21.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:21.081+0000 1694f700 -1 osd.0 271 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:112527 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416060400 gen 1] snapc 0=[] ondisk+write+known_if_redirected e266) 2019-12-08T20:30:22.161 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:30:22.156+0000 1694f700 -1 osd.0 272 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:112656 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 416060400 gen 2] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-12-08T20:30:22.996 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:30:22.977+0000 20e25700 -1 RGWWatcher::handle_error cookie 416060400 err (107) Transport endpoint is not connected 2019-12-08T20:31:39.727 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:39.722+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:40.242 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:40.238+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:40.376 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:40.370+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:40.706 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:40.701+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:40.825 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:40.819+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:41.233 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:41.228+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:41.344 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:41.338+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:41.756 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:41.751+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:41.859 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:41.851+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:42.249 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:42.241+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:42.330 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:42.324+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:42.559 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:42.553+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:42.762 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:42.758+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:42.883 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:42.877+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:42.889 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:42.884+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:43.287 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:43.281+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:43.336 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:43.330+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:43.417 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:43.411+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:43.597 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:43.587+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:43.794 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:43.788+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:43.862 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:43.855+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:43.930 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:43.926+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:44.283 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:44.278+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:44.363 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:44.357+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:44.469 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:44.464+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:44.623 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:44.617+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:44.821 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:44.815+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:44.831 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:44.825+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:44.912 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:44.907+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:45.248 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:45.244+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:45.361 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:45.355+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:45.506 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:45.499+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:45.601 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:45.592+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:45.823 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:45.818+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:45.876 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:45.870+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:45.919 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:45.909+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:46.278 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:46.273+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:46.357 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:46.352+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:46.517 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:46.510+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:46.639 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:46.636+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:46.841 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:46.836+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:46.892 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:46.886+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:46.958 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:46.952+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:47.299 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:47.294+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:47.406 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:47.400+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:47.515 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:47.510+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:47.614 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:47.610+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:47.848 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:47.841+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:47.849 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:47.843+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:47.852 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:47.848+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:47.993 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:47.989+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:48.349 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:48.342+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:48.369 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:48.363+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:48.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:48.533+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:48.575 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:48.571+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:48.813 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:48.805+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:48.818 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:48.813+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:48.819 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:48.814+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:49.013 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:49.009+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:49.330 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:49.326+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:49.405 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:49.399+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:49.505 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:49.498+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:49.570 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:49.566+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:49.801 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:49.796+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:49.836 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:49.830+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:49.837 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:49.831+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:49.998 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:49.995+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:50.374 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:50.370+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:50.453 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:50.447+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:50.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:50.513+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:50.586 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:50.581+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:50.841 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:50.835+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:50.842 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:50.836+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:50.846 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:50.843+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:50.958 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:50.954+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:51.374 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:51.369+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:51.464 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:51.459+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:51.560 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:51.554+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:51.595 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:51.592+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:51.856 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:51.850+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:51.856 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:51.850+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:51.879 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:51.875+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:51.991 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:51.986+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:52.362 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:52.358+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:52.511 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:52.505+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:52.558 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:52.553+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:52.595 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:52.589+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:52.868 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:52.862+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:52.870 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:52.866+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:52.909 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:52.905+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:52.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:52.964+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:53.322 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:53.318+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:53.523 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:53.517+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:53.578 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:53.572+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:53.586 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:53.578+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:53.863 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:53.858+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:53.866 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:53.861+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:53.924 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:53.919+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:53.958 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:53.955+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:54.373 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:54.369+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:54.562 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:54.556+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:54.599 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:54.595+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:54.631 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:54.625+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:54.903 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:54.896+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:54.906 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:54.900+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:54.969 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:54.965+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:55.004 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:54.999+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:55.360 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:55.357+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:55.518 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:55.512+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:55.621 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:55.616+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:55.633 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:55.630+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:55.870 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:55.864+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:55.873 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:55.868+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:55.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:55.969+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:55.977 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:55.973+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:56.325 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:56.321+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:56.485 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:56.479+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:56.640 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:56.636+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:56.671 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:56.664+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:56.869 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:56.864+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:56.871 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:56.867+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:56.945 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:56.936+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:56.983 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:56.974+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:57.298 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:57.295+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:57.525 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:57.519+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:57.634 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:57.628+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:57.648 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:57.644+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:57.860 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:57.855+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:57.862 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:57.858+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:57.953 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:57.948+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:57.956 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:57.952+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:58.333 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:58.329+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:58.512 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:58.506+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:58.616 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:58.610+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:58.693 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:58.689+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:58.844 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:58.835+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:58.848 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:58.842+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:58.925 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:58.922+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:31:58.943 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:58.940+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:59.376 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:31:59.371+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:31:59.509 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:31:59.502+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:31:59.662 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:31:59.655+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:31:59.713 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:31:59.709+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:31:59.887 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:59.881+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:31:59.887 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:31:59.882+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:31:59.940 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:31:59.935+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:31:59.966 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:31:59.963+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:00.366 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:00.359+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:00.495 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:00.489+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:00.669 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:00.664+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:00.699 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:00.693+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:00.866 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:00.859+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:00.867 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:00.861+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:00.926 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:00.923+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:00.944 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:00.939+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:01.378 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:01.375+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:01.528 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:01.522+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:01.676 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:01.670+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:01.678 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:01.674+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:01.827 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:01.821+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:01.828 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:01.822+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:01.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:01.947+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:01.974 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:01.969+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:02.372 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:02.368+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:02.518 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:02.507+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:02.663 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:02.656+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:02.713 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:02.709+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:02.870 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:02.865+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:02.872 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:02.868+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:02.934 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:02.930+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:03.000 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:02.996+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:03.354 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:03.351+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:03.473 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:03.467+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:03.705 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:03.699+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:03.761 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:03.757+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:03.872 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:03.866+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:03.873 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:03.867+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:03.925 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:03.921+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:04.018 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:04.013+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:04.409 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:04.404+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:04.509 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:04.503+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:04.732 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:04.723+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:04.764 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:04.760+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:04.841 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:04.835+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:04.841 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:04.836+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:04.907 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:04.903+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:05.041 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:05.037+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:05.414 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:05.410+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:05.465 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:05.459+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:05.759 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:05.756+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:05.762 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:05.756+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:05.850 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:05.844+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:05.850 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:05.845+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:05.871 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:05.868+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:06.091 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:06.087+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:06.423 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:06.418+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:06.511 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:06.505+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:06.735 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:06.729+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:06.739 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:06.736+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:06.852 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:06.846+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:06.885 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:06.880+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:06.886 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:06.881+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:07.066 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:07.063+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:07.446 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:07.435+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:07.541 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:07.535+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:07.694 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:07.689+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:07.775 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:07.771+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:07.839 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:07.835+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:07.899 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:07.894+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:07.901 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:07.897+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:08.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:08.067+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:08.443 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:08.439+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:08.595 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:08.584+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:08.708 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:08.703+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:08.777 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:08.773+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:08.846 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:08.841+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:08.885 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:08.878+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:08.885 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:08.879+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:09.112 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:09.109+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:09.489 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:09.484+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:09.629 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:09.624+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:09.719 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:09.714+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:09.765 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:09.760+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:09.885 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:09.881+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:09.899 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:09.886+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:09.900 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:09.890+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:10.111 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:10.107+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:10.535 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:10.531+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:10.629 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:10.621+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:10.687 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:10.680+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:10.761 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:10.756+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:10.857 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:10.853+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:10.941 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:10.934+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:10.941 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:10.935+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:11.101 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:11.094+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:11.580 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:11.573+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:11.620 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:11.614+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:11.719 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:11.713+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:11.779 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:11.775+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:11.837 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:11.834+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:11.897 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:11.891+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:11.898 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:11.892+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:12.072 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:12.067+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:12.572 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:12.568+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:12.648 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:12.637+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:12.692 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:12.685+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:12.833 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:12.828+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:12.856 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:12.850+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:12.858 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:12.854+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:12.859 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:12.854+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:13.087 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:13.082+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:13.581 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:13.576+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:13.606 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:13.600+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:13.681 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:13.675+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:13.821 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:13.816+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:13.855 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:13.849+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:13.856 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:13.850+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:13.903 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:13.899+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:14.108 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:14.103+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:14.548 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:14.544+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:14.643 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:14.636+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:14.710 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:14.704+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:14.810 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:14.806+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:14.848 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:14.836+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:14.850 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:14.843+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:14.932 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:14.929+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:15.134 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:15.130+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:15.590 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:15.586+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:15.650 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:15.645+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:15.674 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:32:15.669+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 42sec e276 v276) 2019-12-08T20:32:15.679 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:15.674+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:15.839 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:15.836+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:15.843 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:15.837+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:15.843 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:15.839+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:15.904 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:15.900+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:16.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:16.137+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:16.179 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:48:47.607 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.185 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:48:47.613 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.198 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:48:47.624 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.203 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:48:47.629 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.208 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:48:47.801 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.233 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:48:47.826 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.454 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:48:47.873 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.482 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:48:47.909 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.522 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:48:48.114 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:32:16.616 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:16.612+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:16.704 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:16.699+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:16.721 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:16.717+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:16.834 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:16.830+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:16.892 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:16.888+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:16.895 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:16.891+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:16.935 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:16.931+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:17.179 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:17.175+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:17.626 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:17.623+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:17.709 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:17.704+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:17.751 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:17.746+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:17.809 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:17.805+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:17.861 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:17.856+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:17.862 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:17.857+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:17.891 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:17.888+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:18.198 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:18.194+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:18.614 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:18.609+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:18.709 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:18.705+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:18.743 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:18.737+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:18.807 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:18.801+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:18.857 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:18.852+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:18.858 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:18.853+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:18.858 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:18.854+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:19.209 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:19.205+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:19.575 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:19.571+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:19.709 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:19.704+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:19.732 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:19.727+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:19.828 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:19.825+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:19.839 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:19.833+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:19.839 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:19.834+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:19.885 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:32:19.881+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.629168+0000 front 2019-12-08T20:30:58.600591+0000 (oldest deadline 2019-12-08T20:31:42.520162+0000) 2019-12-08T20:32:20.249 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:32:20.245+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:57.767277+0000 front 2019-12-08T20:30:58.383026+0000 (oldest deadline 2019-12-08T20:31:39.266619+0000) 2019-12-08T20:32:20.615 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:32:20.610+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:59.827970+0000 front 2019-12-08T20:30:57.322649+0000 (oldest deadline 2019-12-08T20:31:40.195339+0000) 2019-12-08T20:32:20.678 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:32:20.673+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:56.039809+0000 front 2019-12-08T20:30:56.101607+0000 (oldest deadline 2019-12-08T20:31:39.513399+0000) 2019-12-08T20:32:20.689 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:20.684+0000 1694f700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.599094+0000 front 2019-12-08T20:30:58.578018+0000 (oldest deadline 2019-12-08T20:31:42.622656+0000) 2019-12-08T20:32:20.746 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:32:20.734+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 42sec e276 v276) 2019-12-08T20:32:20.831 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:32:20.826+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:58.018024+0000 front 2019-12-08T20:30:58.755762+0000 (oldest deadline 2019-12-08T20:31:42.064607+0000) 2019-12-08T20:32:20.858 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:20.850+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:30:55.527117+0000 front 2019-12-08T20:30:55.516580+0000 (oldest deadline 2019-12-08T20:31:40.679075+0000) 2019-12-08T20:32:20.858 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:20.851+0000 1694f700 -1 osd.7 276 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:21.832 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:21.828+0000 1694f700 -1 osd.7 277 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:22.918 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:32:22.864+0000 1694f700 -1 osd.7 278 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:112889 2.1ds0 2:ba91d20b:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%eb%8c%8b%eb%8c%8d%e1%a0%84%ef%84%8f%e5%98%86%e1%a0%83%e0%a4%80%e0%a4%83%e3%9c%86%eb%8c%8e%ed%88%8d%ce%a9%ed%88%8f%e0%a4%81%e0%a4%80:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~7877 in=7877b,setxattr user.rgw.manifest (386) in=403b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e276) 2019-12-08T20:32:50.609 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:50.590+0000 31786700 -1 osd.6 281 build_incremental_map_msg missing incremental map 281 2019-12-08T20:32:50.718 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:50.692+0000 34f8d700 -1 osd.6 281 build_incremental_map_msg missing incremental map 281 2019-12-08T20:32:50.721 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:50.697+0000 34f8d700 -1 osd.6 281 build_incremental_map_msg missing incremental map 281 2019-12-08T20:32:51.554 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:51.466+0000 3378a700 -1 osd.6 281 build_incremental_map_msg missing incremental map 281 2019-12-08T20:32:51.974 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:51.934+0000 31f87700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:51.975 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:51.936+0000 31f87700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:51.990 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:51.970+0000 3578e700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:52.003 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:51.984+0000 36790700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:52.014 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:51.996+0000 32788700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:52.095 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:52.084+0000 3578e700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:52.194 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:52.106+0000 3578e700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:32:54.673 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:32:54.668+0000 38f95700 -1 osd.4 282 build_incremental_map_msg missing incremental map 246 2019-12-08T20:32:54.905 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:54.854+0000 3378a700 -1 osd.6 282 build_incremental_map_msg missing incremental map 281 2019-12-08T20:32:54.908 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:32:54.855+0000 3378a700 -1 osd.6 282 build_incremental_map_msg missing incremental map 282 2019-12-08T20:33:16.420 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:49:47.847 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.422 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:49:47.850 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.437 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:49:48.030 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.485 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:49:47.911 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.488 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:49:47.914 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.493 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:49:47.921 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.495 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:49:48.088 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.599 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:49:48.025 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:16.600 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:49:48.193 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:33:19.323 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:19.319+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:19.375 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:19.368+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:20.151 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:20.144+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:20.327 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:20.321+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:20.360 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:20.353+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:20.602 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:20.596+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:21.111 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:21.102+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:21.237 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:21.229+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:21.309 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:21.305+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:21.348 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:21.340+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:21.653 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:21.648+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:22.025 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:22.022+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:22.163 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:22.156+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:22.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:22.215+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:22.265 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:22.258+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:22.278 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:22.275+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:22.398 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:22.391+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:22.645 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:22.638+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:23.020 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:23.016+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:23.136 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:23.129+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:23.261 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:23.254+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:23.266 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:23.261+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:23.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:23.263+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:23.277 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:23.272+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:23.385 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:23.379+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:23.627 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:23.621+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:24.018 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:24.013+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:24.122 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:24.117+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:24.239 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:24.233+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:24.239 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:24.236+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:24.288 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:24.284+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:24.288 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:24.285+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:24.414 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:24.408+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:24.644 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:24.640+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:24.999 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:24.995+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:25.142 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:25.134+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:25.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:25.202+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:25.254 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:25.247+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:25.333 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:25.328+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:25.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:25.331+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:25.444 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:25.437+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:25.665 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:25.659+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:25.990 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:25.985+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:26.192 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:26.186+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:26.247 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:26.243+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:26.272 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:26.265+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:26.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:26.346+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:26.353 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:26.350+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:26.416 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:26.408+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:26.686 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:26.680+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:26.995 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:26.991+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:27.243 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:27.236+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:27.251 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:27.247+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:27.265 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:27.260+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:27.341 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:27.337+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:27.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:27.340+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:27.388 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:27.382+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:27.667 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:27.659+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:27.668 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:27.662+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:28.000 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:27.996+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:28.202 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:28.195+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:28.233 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:28.226+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:28.256 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:28.251+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:28.327 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:28.322+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:28.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:28.326+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:28.418 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:28.412+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:28.656 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:28.648+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:28.657 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:28.652+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:29.039 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:29.034+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:29.205 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:29.198+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:29.211 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:29.206+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:29.268 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:29.260+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:29.289 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:29.285+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:29.292 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:29.289+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:29.404 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:29.397+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:29.693 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:29.686+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:29.697 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:29.691+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:30.046 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:30.040+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:30.208 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:30.203+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:30.215 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:30.208+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:30.294 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:30.286+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:30.321 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:30.316+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:30.321 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:30.318+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:30.379 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:30.372+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:30.718 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:30.709+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:30.720 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:30.712+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:31.090 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:31.086+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:31.161 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:31.157+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:31.179 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:31.173+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:31.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:31.296+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:31.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:31.298+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:31.318 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:31.311+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:31.404 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:31.397+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:31.740 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:31.732+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:31.741 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:31.735+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:32.057 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:32.051+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:32.146 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:32.141+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:32.214 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:32.207+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:32.284 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:32.280+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:32.284 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:32.281+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:32.300 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:32.292+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:32.367 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:32.360+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:32.744 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:32.737+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:32.747 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:32.742+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:33.071 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:33.068+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:33.106 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:33.102+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:33.240 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:33.233+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:33.307 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:33.300+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:33.332 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:33.329+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:33.335 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:33.332+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:33.403 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:33.396+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:33.769 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:33.761+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:33.770 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:33.764+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:34.104 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:34.099+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:34.141 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:34.120+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:34.222 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:34.214+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:34.271 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:34.264+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:34.302 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:34.296+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:34.304 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:34.299+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:34.398 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:34.391+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:34.804 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:34.796+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:34.805 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:34.799+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:35.116 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:35.112+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:35.134 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:35.130+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:35.258 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:35.249+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:35.322 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:35.315+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:35.341 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:35.337+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:35.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:35.339+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:35.355 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:35.348+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:35.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:35.828+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:35.837 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:35.831+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:36.125 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:36.121+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:36.169 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:36.162+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:36.238 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:36.231+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:36.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:36.301+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:36.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:36.302+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:36.343 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:36.336+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:36.401 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:36.394+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:36.807 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:36.799+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:36.808 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:36.802+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:37.132 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:37.128+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:37.192 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:37.189+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:37.261 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:37.257+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:37.263 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:37.260+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:37.281 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:37.275+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:37.396 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:37.390+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:37.416 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:37.409+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:37.821 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:37.812+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:37.822 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:37.817+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:38.148 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:38.143+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:38.229 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:38.224+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:38.297 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:38.291+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:38.297 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:38.294+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:38.333 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:38.325+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:38.375 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:38.368+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:38.407 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:38.400+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:38.835 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:38.827+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:38.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:38.830+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:39.154 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:39.150+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:39.204 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:39.198+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:39.325 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:39.319+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:39.326 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:39.322+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:39.328 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:39.321+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:39.415 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:39.404+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:39.454 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:39.447+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:39.845 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:39.835+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:39.846 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:39.837+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:40.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:40.202+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:40.244 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:40.239+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:40.287 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:40.283+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:40.288 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:40.284+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:40.341 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:40.333+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:40.368 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:40.361+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:40.486 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:40.479+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:40.840 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:40.834+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:40.841 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:40.836+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:41.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:41.229+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:41.238 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:41.234+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:41.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:41.252+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:41.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:41.254+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:41.360 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:41.353+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:41.419 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:41.412+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:41.466 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:41.459+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:41.858 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:41.850+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:41.859 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:41.853+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:42.244 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:42.240+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:42.290 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:42.285+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:42.296 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:42.292+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:42.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:42.295+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:42.324 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:42.317+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:42.446 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:42.439+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:42.453 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:42.446+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:42.898 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:42.891+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:42.902 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:42.897+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:43.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:43.202+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:43.281 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:43.277+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:43.284 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:43.281+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:43.340 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:43.336+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:43.346 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:43.340+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:43.480 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:43.473+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:43.498 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:43.492+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:43.867 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:43.859+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:43.868 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:43.862+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:44.188 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:44.183+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:44.298 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:44.292+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:44.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:44.296+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:44.383 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:44.379+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:44.389 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:44.383+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:44.471 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:44.463+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:44.490 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:44.482+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:44.896 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:44.888+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:44.897 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:44.891+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:45.167 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:45.162+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:45.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:45.252+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:45.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:45.254+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:45.343 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:45.338+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:45.346 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:45.340+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:45.481 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:45.473+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:45.492 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:45.485+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:45.918 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:45.910+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:45.919 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:45.913+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:46.171 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:46.167+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:46.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:46.227+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:46.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:46.230+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:46.317 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:46.310+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:46.385 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:46.381+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:46.436 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:46.429+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:46.540 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:46.532+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:46.903 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:46.895+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:46.903 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:46.898+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:47.176 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:47.172+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:47.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:47.185+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:47.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:47.187+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:47.339 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:47.332+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:47.359 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:47.355+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:47.426 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:47.419+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:47.523 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:47.516+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:47.902 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:47.894+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:47.903 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:47.897+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:48.161 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:48.157+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:48.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:48.165+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:48.170 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:48.167+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:48.309 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:48.301+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:48.370 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:48.366+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:48.395 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:48.388+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:48.481 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:48.475+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:48.945 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:48.936+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:48.946 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:48.940+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:49.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:49.157+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:49.162 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:49.159+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:49.164 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:49.160+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:49.333 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:49.329+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:49.335 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:49.328+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:49.433 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:49.425+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:49.493 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:49.486+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:49.930 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:49.922+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:49.931 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:49.925+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:50.139 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:50.134+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:50.140 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:50.136+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:50.151 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:50.146+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:50.367 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:50.362+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:50.369 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:50.362+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:50.467 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:50.458+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:50.468 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:50.459+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:50.955 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:50.948+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:50.958 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:50.953+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:51.145 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:51.141+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:51.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:51.144+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:51.167 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:51.164+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:51.231 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:33:51.228+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e282 v282) 2019-12-08T20:33:51.348 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:51.340+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:51.400 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:51.396+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:51.480 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:51.473+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:51.501 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:51.494+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:51.944 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:51.937+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:51.947 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:51.942+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:52.149 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:52.144+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:52.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:52.146+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:52.211 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:52.206+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:52.365 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:52.361+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:52.381 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:52.374+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:52.514 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:52.506+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:52.515 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:52.507+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:52.998 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:52.992+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:53.003 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:52.997+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:53.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:53.159+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:53.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:53.160+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:53.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:53.211+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:53.387 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:53.383+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:53.403 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:53.395+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:53.518 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:53.511+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:53.523 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:53.516+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:53.979 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:53.970+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:53.980 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:53.973+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:54.183 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:54.179+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:54.196 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:54.191+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:54.196 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:54.192+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:54.416 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:54.411+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:54.451 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:54.445+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:54.481 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:54.474+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:54.497 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:54.490+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:54.957 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:54.950+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:54.960 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:54.955+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:55.184 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:55.181+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:55.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:55.196+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:55.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:55.197+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:55.436 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:33:55.429+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.550686+0000 front 2019-12-08T20:32:35.425672+0000 (oldest deadline 2019-12-08T20:33:19.562191+0000) 2019-12-08T20:33:55.442 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:33:55.438+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.644866+0000 front 2019-12-08T20:32:37.649920+0000 (oldest deadline 2019-12-08T20:33:21.430428+0000) 2019-12-08T20:33:55.512 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:55.505+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:55.516 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:55.509+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:55.997 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:55.991+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:32:46.792020+0000 (oldest deadline 2019-12-08T20:33:26.792020+0000) 2019-12-08T20:33:55.998 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:55.993+0000 1694f700 -1 osd.6 282 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:56.161 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:33:56.156+0000 1694f700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:34.810400+0000 front 2019-12-08T20:32:34.780879+0000 (oldest deadline 2019-12-08T20:33:18.545232+0000) 2019-12-08T20:33:56.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:56.194+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:37.385889+0000 front 2019-12-08T20:32:37.379053+0000 (oldest deadline 2019-12-08T20:33:21.375026+0000) 2019-12-08T20:33:56.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:56.196+0000 1694f700 -1 osd.0 282 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:56.272 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:33:56.267+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e282 v282) 2019-12-08T20:33:56.499 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:33:56.489+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:36.667473+0000 front 2019-12-08T20:32:37.072969+0000 (oldest deadline 2019-12-08T20:33:21.194303+0000) 2019-12-08T20:33:56.543 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:33:56.537+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:32:35.467464+0000 front 2019-12-08T20:32:34.824357+0000 (oldest deadline 2019-12-08T20:33:19.165833+0000) 2019-12-08T20:33:56.993 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:56.987+0000 1694f700 -1 osd.6 283 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-08T20:33:57.089 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:57.075+0000 37792700 -1 osd.6 283 build_incremental_map_msg missing incremental map 281 2019-12-08T20:33:57.112 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:33:57.075+0000 37792700 -1 osd.6 283 build_incremental_map_msg missing incremental map 282 2019-12-08T20:33:57.240 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:57.224+0000 1694f700 -1 osd.0 283 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113079 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:58.422 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:33:58.389+0000 1694f700 -1 osd.0 284 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113126 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441172912 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-12-08T20:33:58.939 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:33:58.822+0000 20e25700 -1 RGWWatcher::handle_error cookie 5441172912 err (107) Transport endpoint is not connected 2019-12-08T20:34:31.450 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:34:31.430+0000 30f85700 -1 osd.1 287 build_incremental_map_msg missing incremental map 287 2019-12-08T20:34:31.458 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:34:31.445+0000 30f85700 -1 osd.1 287 build_incremental_map_msg missing incremental map 287 2019-12-08T20:34:31.513 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:34:31.493+0000 35f8f700 -1 osd.1 287 build_incremental_map_msg missing incremental map 287 2019-12-08T20:34:32.582 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:34:32.577+0000 3578e700 -1 osd.1 287 build_incremental_map_msg missing incremental map 287 2019-12-08T20:34:34.844 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:34:34.840+0000 37792700 -1 osd.1 287 build_incremental_map_msg missing incremental map 287 2019-12-08T20:35:30.183 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:30.178+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:31.026 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:31.018+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:31.185 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:31.180+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:31.338 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:31.332+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:31.472 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:31.466+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:32.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:32.069+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:32.158 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:32.154+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:32.372 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:32.364+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:32.454 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:32.446+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:32.707 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:32.702+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:33.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:33.071+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:33.141 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:33.137+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:33.370 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:33.365+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:33.476 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:33.471+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:33.722 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:33.715+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:33.882 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:33.877+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:34.075 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:34.072+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:34.126 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:34.115+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:34.361 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:34.355+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:34.429 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:34.425+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:34.520 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:34.514+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:34.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:34.516+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:34.682 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:34.678+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:34.870 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:34.863+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:35.112 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:35.108+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:35.154 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:35.151+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:35.392 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:35.386+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:35.439 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:35.434+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:35.485 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:35.480+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:35.489 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:35.483+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:35.710 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:35.707+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:35.922 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:35.915+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:36.118 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:36.114+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:36.132 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:36.128+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:36.386 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:36.380+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:36.445 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:36.436+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:36.445 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:36.437+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:36.457 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:36.453+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:36.708 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:36.705+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:36.918 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:36.912+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:37.094 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:37.090+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:37.171 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:37.167+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:37.418 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:37.415+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:37.429 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:37.424+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:37.449 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:37.444+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:37.451 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:37.447+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:37.731 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:37.727+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:37.969 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:37.959+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:38.062 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:38.057+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:38.142 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:38.137+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:38.399 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:38.390+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:38.409 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:38.401+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:38.441 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:38.435+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:38.442 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:38.438+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:38.729 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:38.725+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:38.957 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:38.951+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:39.091 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:39.087+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:39.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:39.112+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:39.366 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:39.362+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:39.395 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:39.392+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:39.468 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:39.463+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:39.468 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:39.464+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:39.752 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:39.748+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:39.976 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:39.971+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:40.138 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:40.132+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:40.144 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:40.140+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:40.327 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:40.322+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:40.421 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:40.417+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:40.450 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:40.445+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:40.450 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:40.445+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:40.750 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:40.746+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:40.989 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:40.984+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:41.156 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:41.152+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:41.161 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:41.158+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:41.304 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:41.299+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:41.421 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:41.417+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:41.470 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:41.465+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:41.471 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:41.466+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:41.807 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:41.801+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:41.966 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:41.961+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:42.140 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:42.136+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:42.143 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:42.138+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:42.327 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:42.322+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:42.467 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:42.463+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:42.510 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:42.505+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:42.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:42.509+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:42.768 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:42.763+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:43.019 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:43.014+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:43.138 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:43.133+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:43.149 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:43.145+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:43.343 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:43.339+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:43.432 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:43.427+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:43.547 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:43.542+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:43.550 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:43.546+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:43.800 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:43.796+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:43.977 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:43.973+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:44.155 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:44.151+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:44.185 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:44.181+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:44.366 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:44.360+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:44.461 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:44.456+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:44.567 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:44.562+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:44.570 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:44.566+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:44.824 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:44.820+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:44.950 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:44.943+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:45.165 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:45.161+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:45.209 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:45.204+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:45.407 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:45.401+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:45.513 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:45.509+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:45.565 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:45.559+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:45.566 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:45.559+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:45.787 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:45.783+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:45.938 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:45.933+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:46.183 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:46.179+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:46.214 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:46.211+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:46.429 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:46.425+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:46.490 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:46.485+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:46.562 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:46.557+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:46.563 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:46.558+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:46.798 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:46.789+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:46.970 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:46.965+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:47.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:47.176+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:47.228 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:47.225+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:47.416 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:47.411+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:47.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:47.515+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:47.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:47.517+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:47.536 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:47.532+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:47.778 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:47.769+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:47.989 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:47.985+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:48.162 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:48.158+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:48.222 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:48.217+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:48.413 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:48.409+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:48.521 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:48.515+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:48.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:48.515+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:48.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:48.516+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:48.736 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:48.732+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:48.977 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:48.970+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:49.206 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:49.203+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:49.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:49.205+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:49.381 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:49.377+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:49.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:49.503+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:49.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:49.504+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:49.521 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:49.516+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:49.773 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:49.769+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:50.017 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:50.011+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:50.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:50.218+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:50.260 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:50.254+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:50.345 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:50.339+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:50.521 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:50.516+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:50.522 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:50.517+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:50.526 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:50.522+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:50.829 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:50.824+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:50.978 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:50.973+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:51.212 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:51.208+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:51.229 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:51.226+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:51.320 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:51.315+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:51.514 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:51.509+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:51.514 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:51.510+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:51.525 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:51.520+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:51.871 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:51.866+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:51.968 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:51.962+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:52.199 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:52.194+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:52.265 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:52.261+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:52.359 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:52.354+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:52.497 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:52.493+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:52.546 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:52.541+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:52.547 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:52.542+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:52.874 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:52.870+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:53.013 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:53.008+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:53.213 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:53.209+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:53.249 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:53.245+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:53.342 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:53.336+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:53.489 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:53.485+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:53.551 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:53.544+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:53.551 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:53.545+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:53.872 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:53.868+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:54.047 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:54.041+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:54.202 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:54.198+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:54.240 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:54.236+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:54.342 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:54.336+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:54.502 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:54.499+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:54.517 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:54.510+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:54.517 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:54.511+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:54.896 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:54.891+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:55.016 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:55.009+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:55.158 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:55.154+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:55.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:55.225+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:55.387 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:55.381+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:55.479 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:55.472+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:55.479 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:55.473+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:55.532 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:55.528+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:55.857 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:55.854+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:56.036 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:56.030+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:56.137 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:56.132+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:56.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:56.252+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:56.355 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:56.349+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:56.459 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:56.453+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:56.460 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:56.454+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:56.519 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:56.515+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:56.867 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:56.863+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:57.002 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:56.996+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:57.102 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:57.098+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:57.241 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:57.237+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:57.370 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:57.364+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:57.464 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:57.458+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:57.464 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:57.459+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:57.524 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:57.520+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:57.895 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:57.892+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:58.013 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:58.007+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:58.113 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:58.108+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:58.248 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:58.244+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:58.412 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:58.406+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:58.498 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:58.494+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:58.515 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:58.509+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:58.520 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:58.511+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:58.899 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:58.895+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:59.024 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:59.017+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:35:59.137 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:35:59.133+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:35:59.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:35:59.234+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:35:59.396 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:35:59.390+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:35:59.481 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:35:59.478+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:35:59.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:59.532+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:35:59.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:35:59.533+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:35:59.888 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:35:59.884+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:35:59.986 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:35:59.979+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:00.158 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:00.153+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:00.261 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:00.256+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:00.374 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:00.368+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:00.440 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:00.436+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:00.509 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:00.499+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:00.509 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:00.501+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:00.874 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:36:00.869+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:36:01.034 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:36:01.028+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:01.160 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:01.156+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:01.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:01.228+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:01.379 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:01.373+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:01.434 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:01.430+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:01.512 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:01.505+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:01.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:01.507+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:01.615 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:36:01.607+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e287 v287) 2019-12-08T20:36:01.910 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:36:01.906+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:36:02.022 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:36:02.016+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:02.132 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:02.128+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:02.271 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:02.268+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:02.399 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:02.394+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:02.417 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:02.408+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:02.495 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:02.489+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:02.496 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:02.490+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:02.900 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:36:02.896+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:36:03.001 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:36:02.994+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:03.141 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:03.137+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:03.254 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:03.250+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:03.370 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:03.363+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:03.408 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:03.403+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:03.467 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:03.460+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:03.470 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:03.465+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:03.904 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:36:03.900+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:36:03.966 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:36:03.961+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:04.175 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:04.171+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:04.233 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:04.229+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:04.400 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:04.393+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:04.440 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:04.435+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:04.453 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:04.446+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:04.453 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:04.447+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:04.901 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:36:04.897+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:36:04.954 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:36:04.948+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:05.174 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:05.170+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:05.223 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:05.220+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:05.352 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:05.347+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:05.448 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:05.445+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:05.484 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:05.478+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:05.487 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:05.482+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:05.870 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:36:05.866+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.749715+0000 front 2019-12-08T20:34:47.675057+0000 (oldest deadline 2019-12-08T20:35:32.126458+0000) 2019-12-08T20:36:05.960 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:36:05.953+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:48.463427+0000 front 2019-12-08T20:34:49.051377+0000 (oldest deadline 2019-12-08T20:35:33.106316+0000) 2019-12-08T20:36:06.158 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:36:06.153+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.747456+0000 front 2019-12-08T20:34:47.894581+0000 (oldest deadline 2019-12-08T20:35:30.016333+0000) 2019-12-08T20:36:06.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:06.224+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.734512+0000 front 2019-12-08T20:34:47.521103+0000 (oldest deadline 2019-12-08T20:35:30.416781+0000) 2019-12-08T20:36:06.351 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:36:06.345+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:45.847554+0000 front 2019-12-08T20:34:46.133811+0000 (oldest deadline 2019-12-08T20:35:31.126111+0000) 2019-12-08T20:36:06.501 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:36:06.496+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.955914+0000 front 2019-12-08T20:34:47.954976+0000 (oldest deadline 2019-12-08T20:35:33.789275+0000) 2019-12-08T20:36:06.538 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:06.532+0000 1694f700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:34:47.061352+0000 front 2019-12-08T20:34:47.060085+0000 (oldest deadline 2019-12-08T20:35:31.283667+0000) 2019-12-08T20:36:06.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:36:06.533+0000 1694f700 -1 osd.4 287 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113340 2.25s0 2:a651926d:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%ef%84%8f%ef%84%92%e2%98%83%ef%84%90%e5%98%85%e9%90%8c%e3%9c%86%e0%a4%83%e1%a0%83%e8%94%88%eb%8c%8c%e3%9c%83%ed%88%90%e5%98%86%eb%8c%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5989 in=5989b,setxattr user.rgw.manifest (387) in=404b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e287) 2019-12-08T20:36:06.674 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:36:06.670+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e287 v287) 2019-12-08T20:36:07.126 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:52:38.553 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.130 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:52:38.558 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.150 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:52:38.742 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.152 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:52:38.745 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.157 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:52:38.583 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.159 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:52:38.585 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.371 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:52:38.962 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.413 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:52:38.841 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:07.416 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:52:38.841 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:36:40.331 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:36:40.304+0000 30f85700 -1 osd.6 292 build_incremental_map_msg missing incremental map 292 2019-12-08T20:36:40.335 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:36:40.319+0000 30f85700 -1 osd.6 292 build_incremental_map_msg missing incremental map 292 2019-12-08T20:36:40.543 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:36:40.489+0000 30f85700 -1 osd.6 292 build_incremental_map_msg missing incremental map 292 2019-12-08T20:36:40.581 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:36:40.492+0000 30f85700 -1 osd.6 292 build_incremental_map_msg missing incremental map 292 2019-12-08T20:36:40.602 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:36:40.503+0000 3378a700 -1 osd.6 292 build_incremental_map_msg missing incremental map 292 2019-12-08T20:36:41.925 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:36:41.906+0000 3378a700 -1 osd.6 293 build_incremental_map_msg missing incremental map 292 2019-12-08T20:36:55.722 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:55.719+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:36:56.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:56.722+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:36:57.747 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:57.743+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:36:58.796 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:58.790+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:36:59.837 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:36:59.833+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:00.815 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:00.810+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:01.864 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:01.858+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:02.000 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:37:02.911 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:02.908+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:03.918 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:03.914+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:04.938 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:04.934+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:05.306 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:05.301+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:05.653 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:05.645+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:05.941 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:05.938+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:06.343 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:06.339+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:06.615 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:06.606+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:06.952 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:06.948+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:06.953 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:06.950+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:07.196 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:07.190+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:07.325 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:07.317+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:07.333 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:07.329+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:07.599 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:07.593+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:08.000 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:07.994+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:08.001 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:07.997+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:08.185 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:08.179+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:08.290 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:08.285+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:08.328 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:08.320+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:08.609 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:08.600+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:09.036 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:09.032+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:09.038 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:09.035+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:09.206 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:09.202+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:09.279 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:09.276+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:09.346 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:09.339+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:09.618 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:09.611+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:09.997 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:09.993+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:10.001 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:09.997+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:10.165 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:10.160+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:10.305 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:10.298+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:10.306 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:10.302+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:10.630 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:10.622+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:11.015 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:11.010+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:11.018 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:11.014+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:11.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:11.211+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:11.264 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:11.260+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:11.312 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:11.306+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:11.646 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:11.639+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:12.006 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:12.002+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:12.007 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:12.004+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:12.070 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:12.061+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:12.198 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:12.195+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:12.264 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:12.260+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:12.372 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:12.351+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:12.673 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:12.665+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:13.041 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:13.033+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:13.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:13.038+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:13.042 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:13.039+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:13.186 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:13.181+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:13.232 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:13.228+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:13.353 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:13.347+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:13.683 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:13.675+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:14.037 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:14.027+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:14.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:14.073+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:14.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:14.075+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:14.158 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:14.154+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:14.235 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:14.228+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:14.280 INFO:teuthology.orchestra.run.smithi129.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:37:14.387 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:14.374+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:14.652 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:14.645+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:14.995 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:14.988+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:15.095 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:15.090+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:15.096 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:15.093+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:15.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:15.154+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:15.262 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:15.258+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:15.426 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:15.419+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:15.625 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:15.619+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:15.955 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:15.949+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:16.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:16.071+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:16.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:16.073+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:16.147 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:16.143+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:16.300 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:16.296+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:16.424 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:16.417+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:16.597 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:16.590+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:16.929 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:16.922+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:16.933 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:16.927+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:17.034 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:17.026+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:17.034 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:17.031+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:17.156 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:17.152+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:17.275 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:17.270+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:17.384 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:17.376+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:17.585 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:17.577+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:17.977 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:17.966+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:17.978 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:17.972+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:18.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:18.069+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:18.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:18.071+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:18.201 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:18.197+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:18.246 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:18.242+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:18.377 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:18.370+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:18.565 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:18.557+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:18.936 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:18.927+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:18.938 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:18.931+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:19.050 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:19.045+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:19.051 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:19.048+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:19.066 INFO:teuthology.orchestra.run.smithi129.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:37:19.210 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:19.206+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:19.261 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:19.256+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:19.391 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:19.384+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:19.524 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:19.516+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:19.900 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:19.892+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:19.902 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:19.895+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:20.024 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:20.021+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:20.027 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:20.024+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:20.224 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:20.219+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:20.297 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:20.293+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:20.403 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:20.395+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:20.541 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:20.533+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:20.919 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:20.910+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:20.927 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:20.913+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:21.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:21.033+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:21.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:21.034+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:21.221 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:21.217+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:21.332 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:21.328+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:21.429 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:21.423+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:21.554 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:21.548+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:21.880 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:21.873+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:21.881 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:21.876+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:22.086 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:22.081+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:22.087 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:22.083+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:22.180 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:22.176+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:22.346 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:22.341+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:22.399 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:22.393+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:22.527 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:22.521+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:22.840 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:22.835+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:22.844 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:22.840+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:23.056 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:23.051+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:23.057 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:23.053+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:23.155 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:23.150+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:23.339 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:23.335+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:23.414 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:23.409+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:23.536 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:23.530+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:23.888 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:23.881+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:23.889 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:23.884+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:24.102 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:24.098+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:24.105 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:24.101+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:24.187 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:24.182+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:24.356 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:24.351+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:24.383 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:24.377+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:24.509 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:24.502+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:24.911 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:24.904+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:24.912 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:24.907+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:25.063 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:25.059+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:25.064 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:25.060+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:25.158 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:25.154+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:25.360 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:25.356+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:25.422 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:25.415+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:25.532 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:25.525+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:25.928 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:25.921+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:25.929 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:25.924+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:26.073 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:26.069+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:26.074 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:26.071+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:26.117 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:26.112+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:26.330 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:26.325+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:26.475 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:26.468+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:26.582 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:26.577+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:26.911 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:26.905+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:26.914 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:26.910+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:27.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:27.073+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:27.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:27.076+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:27.108 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:27.103+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:27.316 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:27.312+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:27.438 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:27.432+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:27.576 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:27.570+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:27.902 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:27.896+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:27.905 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:27.901+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:28.109 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:28.104+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:28.119 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:28.114+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:28.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:28.117+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:28.361 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:28.356+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:28.429 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:28.423+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:28.538 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:28.533+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:28.928 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:28.921+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:28.929 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:28.924+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:29.067 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:29.059+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:29.075 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:29.071+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:29.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:29.072+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:29.415 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:29.410+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:29.416 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:29.411+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:29.575 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:29.565+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:29.896 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:29.889+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:29.897 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:29.892+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:30.072 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:30.068+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:30.115 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:30.110+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:30.115 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:30.112+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:30.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:30.391+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:30.414 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:30.409+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:30.530 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:30.523+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:30.852 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:30.845+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:30.853 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:30.848+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:31.056 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:31.052+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:31.134 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:31.130+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:31.135 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:31.131+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:31.386 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:31.382+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:31.403 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:31.394+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:31.568 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:31.561+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:31.888 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:31.882+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:31.892 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:31.887+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:32.079 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:32.074+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:32.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:32.165+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:32.170 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:32.167+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:32.423 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:32.418+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:32.432 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:32.426+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:32.593 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:32.588+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:32.881 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:32.874+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:32.882 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:32.877+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:33.122 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:33.117+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:33.212 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:33.208+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:33.213 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:33.209+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:33.377 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:33.373+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:33.472 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:33.465+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:33.596 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:33.588+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:33.872 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:33.865+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:33.878 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:33.870+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:34.094 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:34.088+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:34.166 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:34.161+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:34.167 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:34.163+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:34.430 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:34.426+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:34.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:34.442+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:34.587 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:34.580+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:34.896 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:34.888+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:34.899 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:34.892+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:35.128 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:35.124+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:35.130 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:35.127+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:35.131 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:35.128+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:35.417 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:35.410+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:35.436 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:35.431+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:35.588 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:35.581+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:35.887 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:35.879+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:35.888 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:35.882+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:36.126 INFO:teuthology.orchestra.run.smithi129.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:37:36.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:36.148+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:36.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:36.150+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:36.160 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:36.156+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:36.387 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:36.380+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:36.484 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:36.479+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:36.572 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:36.564+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:36.922 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:36.914+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:36.924 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:36.917+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:36.946 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:37:36.942+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e293 v293) 2019-12-08T20:37:37.107 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:37.103+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:37.108 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:37.104+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:37.191 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:37.187+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:37.436 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:37.429+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:37.472 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:37.467+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:37.530 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:37.522+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:37.906 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:37.898+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:37.907 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:37.901+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:38.155 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:38.151+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:38.159 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:38.154+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:38.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:38.156+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:38.450 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:38.443+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:38.461 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:38.453+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:38.558 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:38.551+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:38.653 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:38.645+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:38.872 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:38.865+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:38.875 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:38.870+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:39.203 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:39.199+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:39.204 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:39.201+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:39.205 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:39.200+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:39.420 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:39.413+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:39.444 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:39.440+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:39.553 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:39.545+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:39.647 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:39.640+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:39.920 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:39.913+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:39.924 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:39.919+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:40.217 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:40.211+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:40.219 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:40.214+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:40.222 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:40.218+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:40.395 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:40.388+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:40.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:40.403+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:40.578 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:40.572+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:40.640 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:40.633+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:40.939 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:40.931+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:40.940 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:40.934+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:41.182 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:41.176+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:41.183 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:41.180+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:41.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:41.219+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:41.404 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:41.400+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:41.445 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:41.438+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:41.553 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:41.546+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:41.679 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:41.670+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:41.929 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:41.919+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:41.931 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:41.925+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:41.964 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:37:41.960+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e293 v293) 2019-12-08T20:37:42.201 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:42.198+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:42.226 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:42.221+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:42.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:42.223+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:42.449 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:42.445+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:42.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:42.447+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:42.550 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:42.543+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:42.704 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:42.696+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:42.976 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:42.969+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:42.980 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:42.974+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:43.180 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:43.174+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:43.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:43.251+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:43.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:43.255+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:43.417 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:43.408+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:43.419 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:43.414+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:43.528 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:43.523+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:43.757 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:43.751+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:43.961 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:43.955+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:43.964 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:43.959+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:44.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:44.208+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:44.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:44.293+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:44.304 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:44.299+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:44.429 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:44.422+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:44.447 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:44.443+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:44.538 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:44.532+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:44.716 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:44.709+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:45.004 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:44.998+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:45.005 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:45.000+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:45.208 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:45.200+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:45.263 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:45.257+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:45.264 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:45.259+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:45.444 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:45.437+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:45.479 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:45.473+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:45.551 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:45.544+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:45.771 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:45.764+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:46.011 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:46.003+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:46.012 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:46.006+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:46.189 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:46.182+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:46.276 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:46.269+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:46.276 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:46.271+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:46.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:37:46.441+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.922344+0000 front 2019-12-08T20:36:21.910739+0000 (oldest deadline 2019-12-08T20:37:06.669030+0000) 2019-12-08T20:37:46.457 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:37:46.452+0000 1694f700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.276601+0000 front 2019-12-08T20:36:22.480045+0000 (oldest deadline 2019-12-08T20:37:05.157266+0000) 2019-12-08T20:37:46.536 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:37:46.529+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:20.059307+0000 front 2019-12-08T20:36:19.878604+0000 (oldest deadline 2019-12-08T20:37:04.931285+0000) 2019-12-08T20:37:46.785 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:37:46.779+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.638494+0000 front 2019-12-08T20:36:22.559295+0000 (oldest deadline 2019-12-08T20:37:37.764024+0000) 2019-12-08T20:37:47.038 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:37:47.019+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e293 v293) 2019-12-08T20:37:47.060 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:47.052+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:36:36.231198+0000 (oldest deadline 2019-12-08T20:37:16.231198+0000) 2019-12-08T20:37:47.061 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:37:47.055+0000 1694f700 -1 osd.6 293 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e291) 2019-12-08T20:37:47.198 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:37:47.194+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:21.203612+0000 front 2019-12-08T20:36:21.201940+0000 (oldest deadline 2019-12-08T20:37:07.089981+0000) 2019-12-08T20:37:47.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:47.224+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:36:22.061001+0000 front 2019-12-08T20:36:21.911507+0000 (oldest deadline 2019-12-08T20:37:05.980444+0000) 2019-12-08T20:37:47.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:47.227+0000 1694f700 -1 osd.0 293 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:48.262 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:37:48.258+0000 1694f700 -1 osd.0 294 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:113478 5.5 5:a93a5511:::notify.2:head [watch ping cookie 822332320 gen 1] snapc 0=[] ondisk+write+known_if_redirected e289) 2019-12-08T20:37:49.627 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:37:49.580+0000 20e25700 -1 RGWWatcher::handle_error cookie 822332320 err (107) Transport endpoint is not connected 2019-12-08T20:38:51.526 INFO:teuthology.orchestra.run.smithi129.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:39:16.227 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:16.222+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:16.269 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:16.264+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:17.259 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:17.255+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:17.283 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:17.280+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:17.394 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:17.388+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:17.964 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:17.958+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:18.254 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:18.249+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:18.271 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:18.266+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:18.365 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:18.359+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:18.377 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:18.372+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:18.935 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:18.930+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:19.226 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:19.221+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:19.254 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:19.249+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:19.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:19.345+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:19.398 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:19.393+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:19.413 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:19.407+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:19.917 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:19.912+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:20.205 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:20.197+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:20.275 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:20.270+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:20.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:20.324+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:20.435 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:20.429+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:20.441 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:20.434+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:20.961 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:20.955+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:21.113 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:21.107+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:21.169 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:21.164+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:21.259 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:21.254+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:21.369 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:21.364+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:21.454 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:21.447+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:21.478 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:21.473+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:21.920 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:21.916+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:22.116 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:22.110+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:22.194 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:22.190+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:22.242 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:22.239+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:22.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:22.370+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:22.436 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:22.431+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:22.492 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:22.486+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:22.881 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:22.877+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:23.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:23.116+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:23.234 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:23.230+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:23.239 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:23.235+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:23.406 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:23.403+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:23.446 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:23.439+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:23.546 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:23.540+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:23.870 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:23.867+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:24.094 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:24.087+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:24.249 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:24.243+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:24.271 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:24.268+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:24.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:24.397+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:24.480 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:24.474+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:24.589 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:24.579+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:24.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:24.857+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:25.056 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:25.051+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:25.230 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:25.226+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:25.235 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:25.231+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:25.447 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:25.443+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:25.497 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:25.490+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:25.607 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:25.600+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:25.826 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:25.821+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:26.066 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:26.060+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:26.187 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:26.182+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:26.219 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:26.216+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:26.466 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:26.462+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:26.482 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:26.477+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:26.636 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:26.630+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:26.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:26.857+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:27.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:27.065+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:27.223 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:27.220+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:27.240 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:27.236+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:27.441 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:27.434+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:27.468 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:27.465+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:27.643 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:27.637+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:27.877 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:27.873+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:28.079 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:28.073+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:28.209 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:28.204+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:28.227 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:28.223+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:28.464 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:28.457+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:28.469 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:28.465+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:28.665 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:28.660+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:28.838 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:28.833+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:29.114 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:29.108+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:29.224 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:29.221+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:29.225 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:29.222+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:29.438 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:29.432+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:29.481 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:29.475+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:29.651 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:29.645+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:29.885 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:29.881+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:30.146 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:30.140+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:30.188 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:30.183+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:30.268 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:30.265+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:30.425 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:30.421+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:30.442 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:30.435+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:30.704 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:30.698+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:30.865 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:30.859+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:31.143 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:31.138+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:31.190 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:31.184+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:31.269 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:31.265+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:31.436 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:31.428+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:31.446 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:31.442+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:31.662 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:31.656+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:31.840 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:31.836+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:32.183 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:32.176+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:32.184 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:32.181+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:32.234 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:32.230+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:32.408 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:32.397+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:32.428 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:32.425+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:32.625 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:32.619+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:32.884 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:32.880+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:33.152 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:33.149+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:33.210 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:33.207+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:33.228 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:33.222+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:33.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:33.408+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:33.418 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:33.412+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:33.581 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:33.574+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:33.932 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:33.927+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:34.197 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:34.192+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:34.241 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:34.238+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:34.266 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:34.261+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:34.421 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:34.416+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:34.461 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:34.455+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:34.594 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:34.587+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:34.932 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:39:34.927+0000 1694f700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:32.180340+0000 front 2019-12-08T20:38:33.747489+0000 (oldest deadline 2019-12-08T20:39:17.078182+0000) 2019-12-08T20:39:35.189 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:35.186+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:35.205 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:35.202+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:35.244 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:35.238+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:35.386 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:35.382+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:35.493 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:35.487+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:35.617 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:35.611+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:36.155 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:36.151+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:36.208 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:36.202+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:36.254 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:36.250+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:36.356 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:36.351+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:36.463 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:36.458+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:36.582 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:36.576+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:37.195 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:37.189+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:37.198 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:37.195+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:37.264 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:37.261+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:37.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:37.333+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:37.492 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:37.486+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:37.620 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:37.613+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:38.194 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:38.189+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:38.200 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:38.197+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:38.290 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:38.287+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:38.363 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:38.360+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:38.540 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:38.534+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:38.661 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:38.654+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:39.169 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:39.163+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:39.236 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:39.231+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:39.296 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:39.292+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:39.395 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:39.391+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:39.518 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:39.512+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:39.693 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:39.687+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:40.131 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:40.126+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:40.227 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:40.222+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:40.296 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:40.293+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:40.360 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:40.356+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:40.535 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:40.529+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:40.683 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:40.677+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:41.103 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:41.097+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:41.201 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:41.197+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:41.348 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:41.343+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:41.398 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:41.393+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:41.519 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:41.513+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:41.694 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:41.688+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:42.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:42.065+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:42.215 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:42.211+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:42.393 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:42.389+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:42.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:42.408+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:42.484 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:42.478+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:42.654 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:42.648+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:43.119 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:43.113+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:43.264 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:43.260+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:43.372 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:43.369+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:43.414 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:43.410+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:43.533 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:43.527+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:43.661 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:43.654+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:44.099 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:44.093+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:44.225 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:44.221+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:44.394 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:44.389+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:44.444 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:44.440+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:44.558 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:44.552+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:44.693 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:44.686+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:45.140 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:45.135+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:45.267 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:45.264+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:45.431 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:45.427+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:45.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:45.458+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:45.599 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:45.592+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:45.738 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:45.732+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:46.168 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:46.162+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:46.287 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:46.283+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:46.450 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:46.446+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:46.460 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:46.454+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:46.564 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:46.557+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:46.780 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:46.774+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:47.171 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:47.165+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:47.283 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:47.280+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:47.491 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:47.487+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:47.506 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:47.502+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:47.589 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:47.583+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:47.775 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:47.768+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:48.148 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:48.142+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:48.250 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:48.245+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:48.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:48.448+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:48.487 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:48.483+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:48.564 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:48.557+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:48.755 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:48.748+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:49.134 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:49.128+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:49.246 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:49.243+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:49.442 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:49.437+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:49.536 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:49.530+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:49.539 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:49.535+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:49.745 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:49.738+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:50.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:50.115+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:50.206 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:50.202+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:50.487 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:50.482+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:50.551 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:50.546+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:50.576 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:50.570+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:50.760 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:50.754+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:51.159 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:51.153+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:51.253 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:51.248+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:51.542 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:51.536+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:51.544 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:51.536+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:51.592 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:51.588+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:51.808 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:51.802+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:52.187 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:39:52.181+0000 1694f700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.811645+0000 front 2019-12-08T20:38:36.425414+0000 (oldest deadline 2019-12-08T20:39:20.475172+0000) 2019-12-08T20:39:52.238 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:39:52.235+0000 1694f700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.271794+0000 front 2019-12-08T20:38:32.018871+0000 (oldest deadline 2019-12-08T20:39:16.082070+0000) 2019-12-08T20:39:52.486 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:39:52.479+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 46sec e298 v298) 2019-12-08T20:39:52.561 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:39:52.555+0000 1694f700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.197966+0000 front 2019-12-08T20:38:34.198930+0000 (oldest deadline 2019-12-08T20:39:18.748583+0000) 2019-12-08T20:39:52.562 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:39:52.559+0000 1694f700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:36.812236+0000 front 2019-12-08T20:38:36.813257+0000 (oldest deadline 2019-12-08T20:39:17.557346+0000) 2019-12-08T20:39:52.623 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:39:52.620+0000 1694f700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:31.789975+0000 front 2019-12-08T20:38:36.657914+0000 (oldest deadline 2019-12-08T20:39:16.179273+0000) 2019-12-08T20:39:52.764 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:39:52.759+0000 1694f700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:38:34.308107+0000 front 2019-12-08T20:38:34.114418+0000 (oldest deadline 2019-12-08T20:39:17.243983+0000) 2019-12-08T20:39:59.321 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:56:30.749 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.332 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:56:30.759 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.339 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:56:30.932 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.345 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:56:30.771 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.349 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:56:30.775 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.354 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:56:30.947 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.583 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:56:31.006 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.683 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:56:31.110 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:39:59.692 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:56:31.284 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.784 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:56:34.211 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.796 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:56:34.223 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.829 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:56:34.257 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.847 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:56:34.273 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.849 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:56:34.275 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.865 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:56:34.291 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.874 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:56:34.467 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.880 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:56:34.473 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:02.905 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:56:34.498 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:40:32.617 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:32.474+0000 35f8f700 -1 osd.6 303 build_incremental_map_msg missing incremental map 303 2019-12-08T20:40:32.636 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:32.506+0000 38794700 -1 osd.6 303 build_incremental_map_msg missing incremental map 303 2019-12-08T20:40:33.459 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.394+0000 31786700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.516 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.398+0000 31786700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.521 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.401+0000 31786700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.521 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.408+0000 37792700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.632 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.479+0000 31f87700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.728 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.514+0000 34f8d700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.743 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.521+0000 32788700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:33.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.705+0000 37792700 -1 osd.6 304 build_incremental_map_msg missing incremental map 303 2019-12-08T20:40:33.837 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:33.705+0000 37792700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:34.503 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:34.412+0000 3378a700 -1 osd.6 304 build_incremental_map_msg missing incremental map 303 2019-12-08T20:40:34.505 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:40:34.412+0000 3378a700 -1 osd.6 304 build_incremental_map_msg missing incremental map 304 2019-12-08T20:40:44.811 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:44.808+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:45.840 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:45.834+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:46.869 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:46.864+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:47.874 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:47.869+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:48.914 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:48.910+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:49.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:49.964+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:50.724 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:50.718+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:50.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:50.963+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:51.776 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:51.769+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:51.858 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:51.852+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:51.962 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:51.958+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:52.456 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:52.450+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:52.782 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:52.776+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:52.876 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:52.871+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:52.931 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:52.926+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:52.932 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:52.928+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:53.129 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:53.122+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:53.438 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:53.434+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:53.802 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:53.795+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:53.905 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:53.901+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:53.965 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:53.961+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:53.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:53.965+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:54.114 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:54.107+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:54.435 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:54.431+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:54.839 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:54.833+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:54.862 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:40:54.856+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:40:54.887 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:54.882+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:54.961 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:54.957+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:54.963 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:54.960+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:55.080 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:55.071+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:55.402 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:55.398+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:55.806 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:55.801+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:55.862 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:55.858+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:55.903 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:40:55.898+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:40:56.007 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:56.003+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:56.008 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:56.004+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:56.063 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:56.057+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:56.423 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:56.417+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:56.836 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:56.831+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:56.859 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:56.853+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:56.913 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:40:56.906+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:40:57.046 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:57.042+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:57.047 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:57.043+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:57.092 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:57.085+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:57.426 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:57.423+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:57.883 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:57.876+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:57.908 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:57.904+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:57.922 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:40:57.915+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:40:58.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:58.073+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:58.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:58.075+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:58.091 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:58.083+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:58.441 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:58.437+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:58.877 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:40:58.870+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:40:58.886 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:58.880+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:58.902 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:58.898+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:59.103 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:59.097+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:40:59.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:40:59.100+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:40:59.133 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:40:59.126+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:40:59.394 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:40:59.391+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:40:59.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:40:59.856+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:40:59.892 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:40:59.885+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:40:59.893 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:40:59.885+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:00.103 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:00.099+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:00.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:00.101+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:00.165 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:00.158+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:00.375 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:00.371+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:00.846 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:00.842+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:00.902 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:00.895+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:00.912 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:00.905+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:01.105 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:01.100+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:01.106 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:01.103+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:01.138 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:01.127+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:01.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:01.394+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:01.822 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:01.817+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:01.907 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:01.899+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:01.908 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:01.900+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:02.079 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:02.069+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:02.079 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:02.070+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:02.180 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:02.174+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:02.396 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:02.391+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:02.796 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:02.793+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:02.890 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:02.882+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:02.903 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:02.895+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:03.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:03.072+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:03.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:03.074+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:03.199 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:03.192+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:03.411 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:03.406+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:03.834 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:03.829+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:03.851 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:03.843+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:03.879 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:03.872+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:04.117 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:04.112+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:04.118 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:04.114+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:04.245 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:04.237+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:04.336 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:04.330+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:04.447 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:04.443+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:04.834 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:04.827+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:04.846 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:04.839+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:04.863 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:04.859+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:05.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:05.133+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:05.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:05.134+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:05.241 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:05.231+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:05.310 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:05.304+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:05.411 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:05.407+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:05.821 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:05.814+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:05.832 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:05.825+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:05.907 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:05.903+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:06.117 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:06.100+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:06.118 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:06.114+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:06.248 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:06.240+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:06.307 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:06.302+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:06.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:06.403+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:06.820 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:06.813+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:06.854 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:06.846+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:06.874 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:06.870+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:07.165 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:07.153+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:07.165 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:07.154+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:07.254 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:07.246+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:07.279 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:07.273+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:07.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:07.455+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:07.803 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:07.796+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:07.853 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:07.849+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:07.868 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:07.861+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:08.144 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:08.139+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:08.145 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:08.141+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:08.243 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:08.235+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:08.299 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:08.293+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:08.466 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:08.462+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:08.812 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:08.804+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:08.854 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:08.849+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:08.929 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:08.917+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:09.100 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:09.096+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:09.102 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:09.099+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:09.284 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:09.273+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:09.303 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:09.296+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:09.305 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:09.298+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:09.423 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:09.419+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:09.860 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:09.852+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:09.879 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:09.874+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:09.898 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:09.891+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:10.126 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:10.121+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:10.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:10.126+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:10.246 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:10.239+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:10.315 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:10.308+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:10.318 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:10.313+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:10.438 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:10.434+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:10.830 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:10.823+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:10.900 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:10.893+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:10.905 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:10.900+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:11.178 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:11.173+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:11.178 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:11.175+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:11.251 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:11.243+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:11.347 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:11.339+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:11.348 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:11.342+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:11.404 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:11.400+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:11.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:11.862+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:11.889 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:11.883+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:11.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:11.947+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:12.225 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:12.218+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:12.227 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:12.222+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:12.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:12.224+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:12.342 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:12.334+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:12.346 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:12.341+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:12.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:12.360+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:12.862 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:12.855+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:12.925 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:12.918+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:12.964 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:12.960+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:13.181 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:13.177+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:13.182 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:13.180+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:13.254 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:13.247+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:13.347 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:13.343+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:13.384 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:13.376+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:13.385 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:13.379+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:13.872 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:13.865+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:13.887 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:13.880+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:13.960 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:13.956+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:14.149 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:14.144+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:14.150 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:14.146+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:14.303 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:14.296+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:14.335 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:14.331+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:14.367 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:14.360+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:14.371 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:14.366+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:14.857 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:14.850+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:14.903 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:14.895+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:14.933 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:14.929+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:15.119 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:15.115+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:15.119 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:15.116+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:15.274 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:15.268+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:15.309 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:15.305+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:15.354 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:15.345+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:15.356 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:15.349+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:15.865 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:15.859+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:15.890 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:15.883+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:15.944 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:15.939+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:16.111 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:16.104+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:16.114 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:16.108+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:16.259 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:16.252+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:16.270 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:16.266+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:16.349 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:16.312+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:16.349 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:16.314+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:16.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:16.853+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:16.876 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:16.870+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:16.919 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:16.914+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:17.149 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:17.136+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:17.149 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:17.138+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:17.288 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:17.283+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:17.321 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:17.317+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:17.345 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:17.338+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:17.345 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:17.340+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:17.825 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:17.818+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:17.898 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:17.894+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:17.910 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:17.902+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:18.123 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:18.117+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:18.125 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:18.120+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:18.320 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:18.313+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:18.355 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:18.351+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:18.386 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:18.376+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:18.387 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:18.381+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:18.856 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:18.848+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:18.867 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:18.863+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:18.879 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:18.872+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:19.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:19.122+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:19.131 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:19.126+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:19.360 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:19.352+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:19.361 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:19.355+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:19.375 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:19.367+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:19.387 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:19.382+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:19.892 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:19.882+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:19.896 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:19.893+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:19.897 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:19.887+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:20.176 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:20.170+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:20.177 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:20.173+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:20.388 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:20.380+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:20.438 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:20.430+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:20.439 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:20.435+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:20.439 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:20.433+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:20.872 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:20.868+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:20.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:20.869+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:20.927 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:20.920+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:21.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:21.168+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:21.174 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:21.171+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:21.384 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:21.378+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:21.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:21.403+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:21.444 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:21.436+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:21.449 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:21.439+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:21.854 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:21.849+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:21.855 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:21.850+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:21.889 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:21.883+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:22.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:22.155+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:22.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:22.160+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:22.358 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:22.352+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:22.361 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:22.356+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:22.468 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:22.462+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:22.469 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:22.463+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:22.817 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:22.811+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:22.860 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:22.853+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:22.910 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:22.903+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:22.928 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:41:22.924+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e304 v304) 2019-12-08T20:41:23.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:23.195+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:23.203 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:23.199+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:23.356 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:23.349+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:23.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:23.402+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:23.511 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:23.503+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:23.512 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:23.506+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:23.831 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:23.826+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:23.887 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:23.880+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:23.927 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:23.920+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:24.212 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:24.206+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:24.213 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:24.209+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:24.341 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:24.334+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:24.414 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:24.409+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:24.487 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:24.480+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:24.491 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:24.486+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:24.858 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:24.852+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:24.924 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:24.916+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:24.939 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:24.931+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:25.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:25.224+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:25.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:25.226+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:25.303 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:25.294+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:25.367 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:25.363+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:25.537 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:25.529+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:25.538 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:25.532+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:25.884 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:25.879+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:25.922 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:25.915+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:25.987 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:25.980+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:26.211 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:26.206+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:26.215 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:26.211+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:26.299 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:26.292+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:26.348 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:26.344+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:26.507 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:26.499+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:26.512 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:26.503+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:26.901 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:26.896+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:26.903 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:26.897+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:27.023 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:27.015+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:27.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:27.191+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:27.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:27.193+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:27.311 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:27.304+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:27.312 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:27.307+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:27.487 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:27.478+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:27.487 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:27.481+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:27.907 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:27.902+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:27.929 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:27.923+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:27.939 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:41:27.937+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e304 v304) 2019-12-08T20:41:28.035 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:28.030+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:28.220 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:28.217+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:28.222 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:28.219+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:28.285 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:28.277+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:28.333 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:28.328+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:28.463 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:28.454+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:28.464 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:28.458+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:28.862 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:28.858+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:28.946 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:28.940+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:29.012 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:29.005+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:29.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:29.224+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:29.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:29.226+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:29.294 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:29.287+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:29.308 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:29.304+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:29.515 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:29.508+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:29.516 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:29.511+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:29.838 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:29.834+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:29.968 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:29.961+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:30.026 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:30.018+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:30.198 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:30.193+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:30.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:30.199+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:30.296 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:30.292+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:30.303 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:30.297+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:30.500 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:30.492+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:30.501 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:30.495+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:30.802 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:30.798+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:30.999 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:30.992+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:31.014 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:31.007+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:31.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:31.158+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:31.167 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:31.163+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:31.307 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:31.299+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:31.337 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:31.332+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:31.491 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:31.482+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:31.491 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:31.486+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:31.834 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:31.827+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:31.982 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:31.975+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:32.060 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:32.052+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:32.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:32.204+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:32.214 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:32.210+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:32.304 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:32.297+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:32.331 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:32.324+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:32.538 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:32.530+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:32.539 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:32.533+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:32.795 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:32.789+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:32.953 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:32.946+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:32.955 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:41:32.950+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e304 v304) 2019-12-08T20:41:33.055 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:33.049+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:33.235 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:33.230+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:33.236 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:33.232+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:33.337 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:33.331+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:33.413 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:33.353+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:33.586 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:33.578+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:33.588 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:33.581+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:33.839 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:33.833+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:33.959 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:33.953+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:34.043 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:34.037+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:34.237 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:34.232+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:34.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:34.234+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:34.342 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:34.336+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:34.388 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:34.381+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:34.616 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:34.609+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:34.617 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:34.611+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:34.797 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:34.792+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:34.927 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:34.920+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:35.062 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:35.056+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:35.252 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:35.247+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:35.253 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:35.249+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:35.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:35.358+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:35.414 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:35.408+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:35.625 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:35.618+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:35.626 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:35.621+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:35.760 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:35.755+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:35.983 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:35.977+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:36.025 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:36.019+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:36.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:36.224+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:36.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:36.227+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:36.359 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:36.354+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:36.416 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:36.410+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:36.640 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:36.633+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:36.642 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:36.636+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:36.739 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:36.734+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:36.942 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:36.934+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:37.073 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:37.065+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:37.249 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:37.244+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:37.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:37.247+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:37.399 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:37.394+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:37.400 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:37.394+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:37.658 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:37.650+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:37.659 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:37.654+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:37.729 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:37.723+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:37.948 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:37.940+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:38.034 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:38.029+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:38.073 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:41:38.067+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e304 v304) 2019-12-08T20:41:38.104 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:58:09.531 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.132 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:58:09.560 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.149 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:58:09.575 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.153 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:58:09.744 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.153 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:58:09.577 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.157 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:58:09.748 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.225 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:07:58:09.653 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:38.245+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:38.254 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:38.249+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:38.255 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:07:58:09.848 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.267 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:07:58:09.693 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:41:38.377 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:38.372+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:38.380 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:38.375+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:38.681 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:38.671+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:38.681 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:38.672+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:38.717 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:38.712+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:38.988 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:38.982+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:38.996 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:38.991+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:39.254 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:39.250+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:39.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:39.253+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:39.395 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:39.391+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:39.433 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:39.418+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:39.681 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:39.674+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:39.682 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:39.677+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:39.715 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:39.712+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:39.971 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:39.964+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:40.042 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:40.035+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:40.235 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:40.230+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:40.236 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:40.232+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:40.384 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:40.377+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:40.441 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:40.427+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:40.645 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:40.637+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:40.651 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:40.646+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:40.671 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:40.667+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:40.927 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:40.921+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:41.024 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:41.018+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:41.264 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:41.259+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:41.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:41.261+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:41.369 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:41.360+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:41.473 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:41.461+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:41.639 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:41.635+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:41.677 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:41.671+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:41.679 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:41.674+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:41.975 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:41.968+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:42.005 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:41.995+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:42.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:42.245+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:42.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:42.248+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:42.376 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:42.369+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:42.447 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:41:42.443+0000 1694f700 -1 osd.2 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.163525+0000 front 2019-12-08T20:40:09.162492+0000 (oldest deadline 2019-12-08T20:40:51.848841+0000) 2019-12-08T20:41:42.669 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:41:42.665+0000 1694f700 -1 osd.3 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.279669+0000 front 2019-12-08T20:40:07.799401+0000 (oldest deadline 2019-12-08T20:40:51.367450+0000) 2019-12-08T20:41:42.696 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:42.690+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:42.699 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:42.695+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:42.953 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:41:42.947+0000 1694f700 -1 osd.5 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:06.712809+0000 front 2019-12-08T20:40:07.019747+0000 (oldest deadline 2019-12-08T20:40:49.796876+0000) 2019-12-08T20:41:42.981 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:41:42.975+0000 1694f700 -1 osd.4 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.263671+0000 front 2019-12-08T20:40:09.248447+0000 (oldest deadline 2019-12-08T20:40:54.535915+0000) 2019-12-08T20:41:43.131 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:41:43.127+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e304 v304) 2019-12-08T20:41:43.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:43.263+0000 1694f700 -1 osd.0 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:09.070064+0000 front 2019-12-08T20:40:09.235367+0000 (oldest deadline 2019-12-08T20:40:52.533268+0000) 2019-12-08T20:41:43.270 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:43.265+0000 1694f700 -1 osd.0 304 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:43.371 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:41:43.366+0000 1694f700 -1 osd.7 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:40:07.541187+0000 front 2019-12-08T20:40:06.931644+0000 (oldest deadline 2019-12-08T20:40:52.805899+0000) 2019-12-08T20:41:43.665 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:43.660+0000 1694f700 -1 osd.6 304 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:40:28.763911+0000 (oldest deadline 2019-12-08T20:41:08.763911+0000) 2019-12-08T20:41:43.666 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:41:43.661+0000 1694f700 -1 osd.6 304 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:108580 7.d 7.7fb93d1d (undecoded) ondisk+retry+write+known_if_redirected e302) 2019-12-08T20:41:44.312 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:41:44.309+0000 1694f700 -1 osd.0 305 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:114136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432304768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e300) 2019-12-08T20:41:45.653 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:41:45.642+0000 20e25700 -1 RGWWatcher::handle_error cookie 5432304768 err (107) Transport endpoint is not connected 2019-12-08T20:42:00.733 INFO:teuthology.orchestra.run.smithi129.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:42:01.194 INFO:teuthology.orchestra.run.smithi129.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-12-08T20:43:01.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:01.195+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:01.824 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:01.820+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:02.187 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:02.181+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:02.377 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:02.370+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:02.652 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:02.645+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:02.797 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:02.792+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:03.215 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:03.207+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:03.219 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:03.214+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:03.378 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:03.372+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:03.619 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:03.612+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:03.833 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:03.828+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:04.187 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:04.182+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:04.269 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:04.263+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:04.400 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:04.394+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:04.641 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:04.635+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:04.873 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:04.866+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:04.902 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:04.897+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:05.181 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:05.174+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:05.319 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:05.313+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:05.392 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:05.387+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:05.624 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:05.618+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:05.870 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:05.865+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:05.901 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:05.895+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:06.206 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:06.199+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:06.316 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:06.312+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:06.375 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:06.368+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:06.590 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:06.578+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:06.911 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:06.905+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:06.932 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:06.924+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:06.945 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:06.940+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:07.195 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:07.189+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:07.292 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:07.286+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:07.329 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:07.323+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:07.580 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:07.575+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:07.888 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:07.882+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:07.898 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:07.893+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:07.898 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:07.894+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:08.178 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:08.172+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:08.279 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:08.274+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:08.298 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:08.292+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:08.585 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:08.579+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:08.878 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:08.872+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:08.884 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:08.879+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:08.918 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:08.913+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:09.215 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:09.208+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:09.315 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:09.310+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:09.329 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:09.323+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:09.623 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:09.612+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:09.862 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:09.857+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:09.904 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:09.900+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:09.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:09.940+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:10.216 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:10.209+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:10.292 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:10.289+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:10.365 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:10.360+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:10.644 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:10.637+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:10.893 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:10.889+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:10.898 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:10.891+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:10.907 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:10.902+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:11.226 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:11.220+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:11.292 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:11.288+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:11.381 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:11.376+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:11.682 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:11.670+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:11.882 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:11.878+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:11.889 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:11.883+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:11.943 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:11.939+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:12.250 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:12.246+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:12.262 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:12.256+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:12.420 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:12.412+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:12.661 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:12.650+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:12.856 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:12.852+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:12.909 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:12.904+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:12.972 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:12.968+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:13.250 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:13.247+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:13.253 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:13.248+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:13.431 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:13.425+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:13.700 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:13.694+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:13.898 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:13.894+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:13.906 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:13.902+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:13.985 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:13.981+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:14.273 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:14.264+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:14.302 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:14.296+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:14.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:14.440+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:14.673 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:14.666+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:14.868 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:14.863+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:14.892 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:14.888+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:14.962 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:14.956+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:15.282 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:15.269+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:15.287 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:15.282+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:15.493 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:15.487+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:15.638 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:15.632+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:15.873 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:15.866+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:15.888 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:15.883+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:15.948 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:15.943+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:16.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:16.250+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:16.308 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:16.301+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:16.492 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:16.486+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:16.629 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:16.623+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:16.844 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:16.840+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:16.899 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:16.894+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:16.923 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:16.913+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:17.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:17.211+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:17.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:17.310+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:17.523 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:17.512+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:17.668 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:17.662+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:17.889 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:17.885+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:17.896 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:17.891+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:17.920 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:17.917+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:18.261 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:18.257+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:18.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:18.309+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:18.487 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:18.481+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:18.710 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:18.705+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:18.881 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:18.878+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:18.924 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:18.920+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:18.975 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:18.971+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:19.294 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:19.290+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:19.327 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:19.322+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:19.469 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:19.462+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:19.665 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:19.660+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:19.890 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:19.885+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:19.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:19.942+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:19.974 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:19.970+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:20.311 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:20.306+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:20.331 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:20.325+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:20.450 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:20.430+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:20.636 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:20.630+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:20.846 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:20.841+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:20.964 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:20.959+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:20.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:20.986+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:21.292 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:21.287+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:21.332 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:21.326+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:21.396 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:21.390+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:21.665 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:21.660+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:21.812 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:21.809+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:21.974 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:21.969+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:22.035 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:22.031+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:22.285 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:22.281+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:22.377 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:22.372+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:22.393 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:22.388+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:22.714 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:22.709+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:22.860 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:22.856+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:22.933 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:22.928+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:23.055 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:23.051+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:23.282 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:23.278+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:23.409 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:23.404+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:23.413 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:23.408+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:23.766 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:23.761+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:23.852 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:23.843+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:23.921 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:23.917+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:24.065 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:24.060+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:24.334 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:24.329+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:24.404 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:24.399+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:24.451 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:24.446+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:24.812 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:24.807+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:24.890 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:24.885+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:24.976 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:24.955+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:25.117 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:25.112+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:25.384 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:25.375+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:25.405 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:25.396+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:25.408 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:25.402+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:25.797 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:25.781+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:25.848 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:25.843+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:25.937 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:25.931+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:26.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:26.160+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:26.364 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:26.360+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:26.372 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:26.367+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:26.442 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:26.437+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:26.805 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:26.799+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:26.818 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:26.813+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:26.960 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:26.956+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:27.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:27.156+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:27.374 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:27.371+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:27.381 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:27.375+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:27.460 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:27.455+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:27.804 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:27.800+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:27.829 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:27.819+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:27.941 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:27.929+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:28.213 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:28.207+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:28.359 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:28.356+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:28.395 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:28.389+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:28.485 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:28.480+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:28.796 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:28.793+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:28.852 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:28.847+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:28.957 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:28.953+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:29.170 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:29.165+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:29.384 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:29.380+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:29.448 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:29.440+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:29.452 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:29.447+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:29.791 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:29.786+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:29.862 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:29.857+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:29.963 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:29.959+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:30.181 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:30.175+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:30.376 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:30.372+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:30.411 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:30.404+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:30.467 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:30.461+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:30.805 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:30.800+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:30.842 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:30.836+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:30.985 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:30.981+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:31.216 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:31.212+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:31.369 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:31.366+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:31.394 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:31.388+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:31.422 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:31.416+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:31.826 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:31.821+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:31.850 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:31.841+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:31.956 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:31.951+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:32.253 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:32.249+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:32.369 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:32.365+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:32.369 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:32.362+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:32.455 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:32.449+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:32.823 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:32.817+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:32.892 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:32.888+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:32.997 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:32.989+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:33.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:33.257+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:33.366 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:33.361+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:33.399 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:33.395+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:33.466 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:43:33.460+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 46sec e309 v309) 2019-12-08T20:43:33.478 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:33.470+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:33.829 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:33.822+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:33.892 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:33.888+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:33.967 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:33.962+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:34.249 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:34.244+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:34.365 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:34.360+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:34.388 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:34.385+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:34.467 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:34.461+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:34.788 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:34.783+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:34.928 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:34.924+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:34.947 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:34.942+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:35.222 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:35.218+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:35.381 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:35.375+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:35.421 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:35.415+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:35.464 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:35.450+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:35.822 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:35.816+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:35.942 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:35.939+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:35.953 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:35.949+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:36.215 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:36.211+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:36.397 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:36.391+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:36.449 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:36.445+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:36.498 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:36.492+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:36.835 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:36.829+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:36.942 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:36.938+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:36.982 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:36.978+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:37.233 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:37.224+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:37.421 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:37.418+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:37.445 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:37.435+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:37.474 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:37.468+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:37.798 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:43:37.792+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.744562+0000 front 2019-12-08T20:42:20.906880+0000 (oldest deadline 2019-12-08T20:43:02.187667+0000) 2019-12-08T20:43:37.926 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:43:37.922+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.858074+0000 front 2019-12-08T20:42:16.303274+0000 (oldest deadline 2019-12-08T20:43:01.554986+0000) 2019-12-08T20:43:38.032 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:43:38.028+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.749828+0000 front 2019-12-08T20:42:21.314702+0000 (oldest deadline 2019-12-08T20:43:04.586561+0000) 2019-12-08T20:43:38.189 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:43:38.185+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.568474+0000 front 2019-12-08T20:42:21.218878+0000 (oldest deadline 2019-12-08T20:43:06.698375+0000) 2019-12-08T20:43:38.379 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:43:38.375+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:21.855612+0000 front 2019-12-08T20:42:16.535243+0000 (oldest deadline 2019-12-08T20:43:01.154867+0000) 2019-12-08T20:43:38.410 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:43:38.405+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:19.364657+0000 front 2019-12-08T20:42:21.858002+0000 (oldest deadline 2019-12-08T20:43:01.554610+0000) 2019-12-08T20:43:38.468 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:43:38.463+0000 1694f700 -1 osd.4 309 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:42:20.819050+0000 front 2019-12-08T20:42:20.821096+0000 (oldest deadline 2019-12-08T20:43:02.291712+0000) 2019-12-08T20:43:38.529 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:43:38.522+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 46sec e309 v309) 2019-12-08T20:43:53.529 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:00:24.957 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.535 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:00:24.963 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.544 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:00:25.136 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.547 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:00:25.140 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.554 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:00:24.980 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.558 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:00:24.984 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.707 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:00:25.130 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.707 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:00:25.296 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:43:53.728 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:00:25.156 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:44:36.690 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:36.685+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:37.177 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:37.161+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:37.508 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:37.503+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:37.707 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:37.700+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:38.192 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:38.183+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:38.554 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:38.549+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:38.704 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:38.697+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:38.892 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:38.885+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:39.183 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:39.175+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:39.363 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:39.358+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:39.523 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:39.518+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:39.723 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:39.717+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:39.883 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:39.876+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:40.212 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:40.205+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:40.362 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:40.356+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:40.491 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:40.483+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:40.543 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:40.535+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:40.772 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:40.766+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:40.905 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:40.891+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:41.218 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:41.211+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:41.380 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:41.375+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:41.538 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:41.529+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:41.562 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:41.557+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:41.789 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:41.785+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:41.869 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:41.862+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:42.178 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:42.172+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:42.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:42.368+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:42.507 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:42.499+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:42.586 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:42.581+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:42.779 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:42.774+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:42.837 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:42.827+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:43.151 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:43.144+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:43.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:43.409+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:43.531 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:43.524+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:43.579 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:43.575+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:43.797 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:43.790+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:43.824 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:43.819+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:44.118 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:44.111+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:44.373 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:44.366+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:44.373 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:44.369+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:44.526 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:44.518+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:44.631 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:44.627+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:44.844 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:44.833+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:44.862 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:44.858+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:45.172 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:45.165+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:45.388 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:45.383+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:45.388 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:45.385+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:45.528 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:45.521+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:45.608 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:45.604+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:45.821 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:45.815+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:45.887 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:45.883+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:46.154 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:46.147+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:46.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:46.407+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:46.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:46.409+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:46.515 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:46.507+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:46.614 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:46.610+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:46.824 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:46.817+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:46.883 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:46.878+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:47.155 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:47.148+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:47.327 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:47.317+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:47.383 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:47.379+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:47.386 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:47.382+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:47.532 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:47.526+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:47.568 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:47.564+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:47.831 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:47.824+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:47.846 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:47.840+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:48.151 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:48.143+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:48.339 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:48.333+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:48.341 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:48.335+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:48.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:48.338+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:48.524 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:48.518+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:48.587 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:48.579+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:48.821 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:48.809+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:48.875 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:48.871+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:49.203 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:49.197+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:49.315 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:49.309+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:49.316 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:49.309+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:49.316 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:49.312+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:49.516 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:49.512+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:49.626 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:49.619+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:49.778 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:49.770+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:49.858 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:49.854+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:50.175 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:50.167+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:50.284 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:50.280+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:50.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:50.283+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:50.367 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:50.360+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:50.558 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:50.553+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:50.667 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:50.660+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:50.780 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:50.772+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:50.822 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:50.819+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:51.211 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:51.204+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:51.312 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:51.308+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:51.313 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:51.309+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:51.373 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:51.367+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:51.599 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:51.594+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:51.672 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:51.665+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:51.793 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:51.786+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:51.795 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:51.790+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:52.265 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:52.257+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:52.301 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:52.296+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:52.302 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:52.298+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:52.345 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:52.339+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:52.628 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:52.622+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:52.706 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:52.700+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:52.768 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:52.760+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:52.794 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:52.789+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:53.236 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:53.229+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:53.275 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:53.267+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:53.276 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:53.270+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:53.363 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:53.356+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:53.366 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:53.358+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:53.631 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:53.627+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:53.695 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:53.687+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:53.727 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:53.720+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:53.837 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:53.832+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:54.264 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:54.260+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:54.265 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:54.262+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:54.286 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:54.279+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:54.384 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:54.376+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:54.385 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:54.377+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:54.594 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:54.589+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:54.703 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:54.695+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:54.733 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:54.727+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:54.792 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:54.787+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:55.250 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:55.246+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:55.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:55.248+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:55.265 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:55.257+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:55.349 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:55.342+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:55.349 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:55.344+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:55.584 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:55.580+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:55.670 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:55.662+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:55.772 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:55.768+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:55.781 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:55.773+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:56.261 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:56.254+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:56.283 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:56.279+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:56.286 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:56.283+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:56.325 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:56.318+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:56.326 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:56.320+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:56.575 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:56.570+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:56.715 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:56.709+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:56.801 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:56.797+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:56.834 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:56.827+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:57.270 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:57.263+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:57.282 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:57.278+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:57.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:57.282+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:57.330 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:57.323+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:57.334 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:57.328+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:57.549 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:57.537+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:57.683 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:57.675+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:57.770 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:57.765+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:57.790 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:57.783+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:58.275 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:58.270+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:58.276 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:58.273+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:58.291 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:58.284+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:58.336 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:58.327+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:58.337 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:58.331+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:58.518 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:58.514+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:58.695 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:58.687+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:58.740 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:58.735+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:44:58.748 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:58.741+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:59.257 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:44:59.251+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:44:59.298 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:59.294+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:44:59.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:44:59.295+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:44:59.387 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:59.380+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:44:59.388 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:44:59.382+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:44:59.552 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:44:59.548+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:44:59.684 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:44:59.673+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:44:59.734 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:44:59.727+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:44:59.734 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:44:59.731+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:00.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:00.254+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:00.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:00.257+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:00.284 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:00.277+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:00.383 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:00.377+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:00.386 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:00.381+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:00.596 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:00.591+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:00.679 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:00.671+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:00.708 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:00.703+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:00.708 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:00.701+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:01.296 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:01.291+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:01.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:01.296+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:01.307 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:01.301+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:01.354 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:01.347+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:01.355 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:01.350+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:01.642 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:01.635+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:01.642 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:01.639+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:01.673 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:01.668+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:01.693 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:01.690+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:02.266 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:02.260+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:02.307 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:02.301+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:02.309 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:02.304+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:02.369 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:02.362+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:02.370 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:02.365+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:02.655 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:02.649+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:02.673 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:02.667+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:02.679 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:02.676+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:02.693 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:02.690+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:03.272 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:03.266+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:03.340 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:03.336+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:03.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:03.339+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:03.396 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:03.389+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:03.396 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:03.392+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:03.668 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:03.661+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:03.682 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:03.676+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:03.689 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:03.684+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:03.713 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:03.708+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:04.279 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:04.273+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:04.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:04.300+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:04.306 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:04.303+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:04.350 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:04.344+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:04.352 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:04.348+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:04.664 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:04.658+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:04.667 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:04.663+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:04.705 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:04.702+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:04.724 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:04.717+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:05.333 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:05.326+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:05.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:05.333+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:05.338 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:05.335+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:05.377 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:05.371+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:05.381 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:05.376+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:05.711 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:05.706+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:05.718 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:05.713+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:05.730 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:05.725+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:05.752 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:05.748+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:06.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:06.337+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:06.343 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:06.340+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:06.348 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:06.342+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:06.350 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:06.345+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:06.366 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:06.361+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:06.687 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:06.681+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:06.694 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:06.687+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:06.702 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:06.698+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:06.789 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:06.784+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:07.309 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:07.303+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:07.313 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:07.308+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:07.353 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:07.348+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:07.364 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:07.357+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:07.364 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:07.361+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:07.697 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:07.693+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:07.698 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:07.692+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:07.723 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:07.718+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:07.744 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:07.738+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:08.298 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:08.292+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:08.302 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:08.298+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:08.311 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:08.306+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:08.390 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:08.384+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:08.390 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:08.386+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:08.704 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:08.697+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:08.721 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:08.715+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:08.721 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:08.717+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:08.768 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:08.764+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:09.167 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:45:09.163+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e315 v315) 2019-12-08T20:45:09.264 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:09.258+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:09.265 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:09.260+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:09.344 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:09.339+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:09.345 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:09.341+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:09.367 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:09.361+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:09.681 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:09.674+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:09.716 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:09.705+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:09.774 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:09.769+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:09.809 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:09.804+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:10.288 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:10.281+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:10.289 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:10.284+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:10.332 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:10.325+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:10.381 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:10.377+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:10.384 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:10.378+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:10.656 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:10.650+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:10.754 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:10.747+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:10.816 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:10.811+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:10.832 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:10.828+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:11.341 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:11.327+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:11.352 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:11.339+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:11.354 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:11.349+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:11.423 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:11.418+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:11.424 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:11.421+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:11.627 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:11.620+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:11.778 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:11.771+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:11.845 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:11.841+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:11.847 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:11.843+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:12.312 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:12.306+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:12.315 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:12.311+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:12.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:12.379+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:12.391 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:12.385+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:12.392 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:12.388+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:12.606 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:12.600+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:12.767 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:12.761+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:12.807 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:12.803+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:12.836 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:12.831+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:13.306 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:13.299+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:13.307 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:13.302+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:13.356 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:13.350+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:13.380 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:13.376+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:13.382 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:13.379+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:13.626 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:13.621+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:13.761 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:13.756+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:13.836 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:13.831+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:13.848 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:13.842+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:14.183 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:45:14.179+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e315 v315) 2019-12-08T20:45:14.317 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:14.310+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:14.318 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:14.311+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:14.318 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:14.312+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:14.377 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:14.372+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:14.378 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:14.375+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:14.608 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:14.601+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:14.748 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:14.741+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:14.846 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:14.842+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:14.855 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:14.851+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:15.300 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:15.292+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:15.315 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:15.308+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:15.317 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:15.311+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:15.404 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:15.399+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:15.404 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:15.401+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:15.592 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:15.585+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:15.720 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:15.714+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:15.820 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:15.816+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:15.909 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:15.902+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:16.304 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:16.297+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:16.365 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:16.358+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:16.368 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:16.363+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:16.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:16.428+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:16.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:16.431+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:16.559 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:16.551+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:16.732 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:16.725+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:16.785 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:16.781+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:16.958 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:16.954+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:17.355 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:17.347+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:17.404 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:17.396+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:17.406 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:17.400+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:17.438 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:17.434+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:17.439 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:17.436+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:17.606 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:17.599+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:17.772 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:17.766+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:17.819 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:17.814+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:17.919 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:17.914+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:18.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:18.378+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:18.395 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:18.391+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:18.397 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:18.394+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:18.421 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:18.415+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:18.422 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:18.416+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:18.650 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:18.643+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:18.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:18.784+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:18.866 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:18.862+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:18.970 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:18.966+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:19.209 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:45:19.204+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e315 v315) 2019-12-08T20:45:19.355 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:19.351+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:19.356 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:19.352+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:19.359 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:19.352+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:19.403 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:19.397+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:19.405 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:19.400+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:19.618 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:19.611+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:19.755 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:19.748+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:19.919 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:19.913+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:20.000 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:19.994+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:20.351 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:20.343+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:20.365 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:20.360+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:20.366 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:20.361+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:20.428 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:20.421+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:20.428 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:20.423+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:20.604 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:20.597+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:20.802 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:20.796+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:20.908 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:20.903+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:21.019 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:21.015+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:21.369 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:21.362+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:21.389 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:21.382+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:21.393 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:21.387+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:21.407 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:21.401+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:21.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:21.405+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:21.588 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:21.580+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:21.828 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:21.821+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:21.889 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:21.885+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:22.009 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:22.004+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:22.378 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:22.371+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:22.380 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:22.372+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:22.381 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:22.375+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:22.433 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:22.425+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:22.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:22.429+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:22.617 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:22.609+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:22.819 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:22.812+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:22.893 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:22.888+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:23.043 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:23.037+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:23.338 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:23.329+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:44:12.601781+0000 (oldest deadline 2019-12-08T20:44:52.601781+0000) 2019-12-08T20:45:23.340 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:23.333+0000 1694f700 -1 osd.6 315 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:23.387 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:23.380+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:23.399 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:23.394+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:23.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:23.396+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:23.650 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:45:23.642+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:54.331898+0000 front 2019-12-08T20:43:54.419365+0000 (oldest deadline 2019-12-08T20:44:39.611127+0000) 2019-12-08T20:45:23.850 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:45:23.843+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:53.724851+0000 front 2019-12-08T20:43:53.733398+0000 (oldest deadline 2019-12-08T20:44:38.409528+0000) 2019-12-08T20:45:23.913 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:45:23.907+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.995868+0000 front 2019-12-08T20:43:56.054883+0000 (oldest deadline 2019-12-08T20:44:37.084042+0000) 2019-12-08T20:45:24.050 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:45:24.045+0000 1694f700 -1 osd.3 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.132824+0000 front 2019-12-08T20:43:55.231870+0000 (oldest deadline 2019-12-08T20:44:36.153493+0000) 2019-12-08T20:45:24.238 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:45:24.233+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e315 v315) 2019-12-08T20:45:24.414 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:45:24.409+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:51.075374+0000 front 2019-12-08T20:43:51.105184+0000 (oldest deadline 2019-12-08T20:44:36.757051+0000) 2019-12-08T20:45:24.433 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:24.428+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:43:55.761531+0000 front 2019-12-08T20:43:55.974160+0000 (oldest deadline 2019-12-08T20:44:38.938415+0000) 2019-12-08T20:45:24.433 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:24.429+0000 1694f700 -1 osd.0 315 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:24.620 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:45:24.429+0000 1694f700 -1 osd.6 316 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e313) 2019-12-08T20:45:25.459 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:45:25.448+0000 1694f700 -1 osd.0 316 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:114806 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5441221072 gen 1] snapc 0=[] ondisk+write+known_if_redirected e312) 2019-12-08T20:45:26.574 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:45:26.562+0000 20e25700 -1 RGWWatcher::handle_error cookie 5441221072 err (107) Transport endpoint is not connected 2019-12-08T20:45:52.987 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:45:52.973+0000 34f8d700 -1 osd.1 320 build_incremental_map_msg missing incremental map 320 2019-12-08T20:45:53.083 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:45:53.012+0000 34f8d700 -1 osd.1 320 build_incremental_map_msg missing incremental map 320 2019-12-08T20:45:56.587 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:45:56.511+0000 36f91700 -1 osd.1 320 build_incremental_map_msg missing incremental map 320 2019-12-08T20:46:52.302 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:52.296+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:52.786 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:52.781+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:53.075 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:53.069+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:53.322 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:53.317+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:53.781 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:53.777+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:54.110 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:54.104+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:54.299 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:54.294+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:54.791 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:54.786+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:55.040 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:46:55.035+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:46:55.071 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:46:55.066+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:46:55.104 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:55.098+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:55.229 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:46:55.224+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:46:55.264 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:55.259+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:55.747 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:55.743+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:56.049 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:46:56.044+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:46:56.057 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:46:56.052+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:46:56.158 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:56.153+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:56.235 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:46:56.228+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:46:56.315 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:56.310+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:56.740 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:56.733+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:57.040 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:46:57.034+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:46:57.108 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:46:57.101+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:46:57.136 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:57.131+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:57.286 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:46:57.280+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:46:57.295 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:57.290+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:57.538 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:46:57.534+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:46:57.740 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:57.734+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:58.069 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:46:58.065+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:46:58.100 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:46:58.094+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:46:58.114 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:58.110+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:58.266 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:46:58.262+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:46:58.296 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:58.292+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:58.502 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:46:58.499+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:46:58.781 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:58.777+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:46:59.088 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:46:59.081+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:46:59.093 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:46:59.087+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:46:59.163 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:46:59.155+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:46:59.286 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:46:59.282+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:46:59.346 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:46:59.343+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:46:59.489 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:46:59.485+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:46:59.739 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:46:59.735+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:00.045 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:00.039+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:00.108 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:00.101+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:00.160 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:00.154+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:00.306 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:00.303+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:00.326 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:00.323+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:00.513 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:00.508+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:00.732 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:00.728+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:01.093 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:01.087+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:01.152 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:01.146+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:01.205 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:01.200+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:01.322 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:01.317+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:01.374 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:01.370+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:01.509 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:01.506+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:01.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:01.721+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:02.149 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:02.139+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:02.180 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:02.174+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:02.212 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:02.207+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:02.309 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:02.304+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:02.419 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:02.414+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:02.489 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:02.485+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:02.714 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:02.710+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:03.169 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:03.164+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:03.174 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:03.169+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:03.180 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:03.175+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:03.306 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:03.303+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:03.427 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:03.423+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:03.489 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:03.485+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:03.765 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:03.761+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:04.158 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:04.153+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:04.174 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:04.169+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:04.201 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:04.197+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:04.331 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:04.327+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:04.456 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:04.451+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:04.468 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:04.464+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:04.765 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:04.761+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:05.145 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:05.139+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:05.166 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:05.161+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:05.198 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:05.192+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:05.312 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:05.306+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:05.458 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:05.453+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:05.510 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:05.502+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:05.733 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:05.729+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:06.132 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:06.126+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:06.158 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:06.152+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:06.200 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:06.193+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:06.281 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:06.277+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:06.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:06.455+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:06.530 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:06.526+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:06.728 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:06.724+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:07.139 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:07.133+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:07.208 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:07.202+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:07.248 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:07.242+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:07.310 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:07.305+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:07.479 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:07.475+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:07.559 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:07.554+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:07.686 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:07.682+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:08.110 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:08.103+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:08.115 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:08.108+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:08.167 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:08.161+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:08.241 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:08.235+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:08.327 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:08.323+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:08.465 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:08.462+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:08.570 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:08.564+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:08.644 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:08.640+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:09.071 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:09.064+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:09.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:09.065+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:09.184 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:09.178+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:09.253 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:09.247+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:09.381 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:09.377+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:09.469 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:09.465+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:09.610 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:09.605+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:09.613 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:09.610+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:10.078 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:10.071+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:10.083 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:10.075+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:10.203 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:10.196+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:10.270 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:10.264+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:10.344 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:10.340+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:10.443 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:10.439+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:10.615 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:10.611+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:10.656 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:10.652+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:11.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:11.124+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:11.131 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:11.125+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:11.196 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:11.189+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:11.276 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:11.270+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:11.365 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:11.360+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:11.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:11.455+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:11.609 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:11.605+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:11.620 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:47:11.616+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.127165+0000 front 2019-12-08T20:46:11.462688+0000 (oldest deadline 2019-12-08T20:46:56.951656+0000) 2019-12-08T20:47:12.135 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:12.129+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:12.136 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:12.130+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:12.213 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:12.206+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:12.276 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:12.270+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:12.397 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:12.392+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:12.487 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:12.483+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:12.652 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:12.647+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:13.144 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:13.137+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:13.144 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:13.139+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:13.255 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:13.248+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:13.257 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:13.252+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:13.449 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:13.446+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:13.451 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:13.445+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:13.674 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:13.670+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:14.118 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:14.112+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:14.119 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:14.113+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:14.233 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:14.227+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:14.287 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:14.281+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:14.444 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:14.438+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:14.476 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:14.469+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:14.648 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:14.644+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:15.078 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:15.072+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:15.079 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:15.073+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:15.276 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:15.269+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:15.308 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:15.302+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:15.460 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:15.456+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:15.492 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:15.488+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:15.697 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:15.692+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:16.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:16.065+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:16.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:16.066+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:16.305 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:16.299+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:16.333 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:16.326+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:16.455 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:16.442+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:16.476 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:16.472+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:16.737 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:16.733+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:17.113 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:17.106+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:17.115 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:17.107+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:17.312 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:17.307+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:17.348 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:17.342+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:17.418 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:17.415+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:17.450 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:17.446+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:17.701 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:17.698+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:18.092 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:18.085+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:18.094 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:18.089+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:18.288 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:18.282+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:18.315 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:18.309+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:18.418 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:18.414+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:18.458 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:18.453+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:18.699 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:18.691+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:19.082 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:19.077+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:19.084 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:19.079+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:19.283 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:19.277+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:19.313 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:19.308+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:19.446 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:19.442+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:19.479 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:19.476+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:19.745 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:19.741+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:20.116 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:20.111+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:20.120 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:20.114+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:20.283 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:20.277+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:20.325 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:20.319+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:20.440 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:20.435+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:20.468 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:20.465+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:20.758 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:20.754+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:21.114 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:21.108+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:21.115 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:21.109+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:21.260 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:21.254+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:21.323 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:21.316+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:21.400 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:21.396+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:21.448 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:21.445+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:21.767 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:21.763+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:22.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:22.115+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:22.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:22.116+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:22.254 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:22.248+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:22.297 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:22.291+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:22.384 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:22.380+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:22.458 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:22.453+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:22.770 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:22.766+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:23.103 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:23.096+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:23.103 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:23.097+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:23.242 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:23.235+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:23.342 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:23.336+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:23.365 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:23.361+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:23.506 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:23.502+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:23.742 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:23.738+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:24.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:24.065+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:24.072 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:24.067+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:24.278 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:24.272+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:24.320 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:24.315+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:24.382 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:24.378+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:24.487 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:24.483+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:24.697 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:47:24.691+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e320 v320) 2019-12-08T20:47:24.711 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:24.707+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:25.108 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:25.102+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:25.109 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:25.103+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:25.251 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:25.246+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:25.318 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:25.312+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:25.400 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:25.397+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:25.538 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:25.534+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:25.714 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:25.710+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:26.089 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:26.083+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:26.089 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:26.084+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:26.252 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:26.245+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:26.345 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:26.339+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:26.422 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:26.418+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:26.520 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:26.515+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:26.724 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:26.720+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:27.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:27.124+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:27.133 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:27.128+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:27.263 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:27.257+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:27.328 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:27.321+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:27.410 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:27.405+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:27.507 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:27.503+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:27.772 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:27.761+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:28.157 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:28.134+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:28.162 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:28.155+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:28.282 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:28.277+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:28.333 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:28.328+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:28.418 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:28.414+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:28.462 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:28.458+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:28.726 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:28.722+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:29.149 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:29.143+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:07.558235+0000 front 2019-12-08T20:46:06.920920+0000 (oldest deadline 2019-12-08T20:46:52.791149+0000) 2019-12-08T20:47:29.149 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:29.143+0000 1694f700 -1 osd.7 320 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:29.331 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:47:29.325+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:10.926872+0000 front 2019-12-08T20:46:11.198423+0000 (oldest deadline 2019-12-08T20:46:54.418199+0000) 2019-12-08T20:47:29.378 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:47:29.372+0000 1694f700 -1 osd.4 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:09.398194+0000 front 2019-12-08T20:46:10.689793+0000 (oldest deadline 2019-12-08T20:46:54.470797+0000) 2019-12-08T20:47:29.406 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:47:29.402+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:08.849394+0000 front 2019-12-08T20:46:09.010602+0000 (oldest deadline 2019-12-08T20:46:52.197583+0000) 2019-12-08T20:47:29.472 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:47:29.467+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.033999+0000 front 2019-12-08T20:46:10.674451+0000 (oldest deadline 2019-12-08T20:46:54.702517+0000) 2019-12-08T20:47:29.693 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:47:29.690+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:46:11.247761+0000 front 2019-12-08T20:46:11.499592+0000 (oldest deadline 2019-12-08T20:46:52.693787+0000) 2019-12-08T20:47:29.738 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:47:29.732+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e320 v320) 2019-12-08T20:47:30.276 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:30.156+0000 1694f700 -1 osd.7 321 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:31.484 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:47:31.465+0000 1694f700 -1 osd.7 322 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115220 2.1ds0 2:ba43d916:::9161132f-da48-40b4-a853-f341a1971d48.4507.610_%eb%8c%8d%e8%94%89%e0%a4%82%e9%90%89%ed%88%8d%e8%94%89%e9%90%8a%e1%a0%82%eb%8c%8c%e0%a4%80%e9%90%8b%e8%94%8b%e0%a4%83%ed%88%8f%e5%98%85:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e320) 2019-12-08T20:47:44.919 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:04:16.344 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:44.923 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:04:16.351 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:44.931 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:04:16.357 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:44.934 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:04:16.527 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:44.935 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:04:16.361 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:44.942 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:04:16.535 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:45.201 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:04:16.629 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:45.204 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:04:16.630 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:47:45.248 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:04:16.840 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:48:14.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:14.008+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:15.038 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:15.032+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:15.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:15.986+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:17.007 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:17.000+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:17.995 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:17.991+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:19.042 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:19.039+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:20.094 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:20.091+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:21.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:21.101+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:22.106 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:22.103+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:23.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:23.073+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:24.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:24.115+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:25.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:25.147+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:26.149 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:26.145+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:27.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:27.158+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:28.124 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:28.118+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:28.196 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:28.192+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:28.197 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:28.190+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:29.087 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:29.082+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:29.156 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:29.149+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:29.156 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:29.152+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:29.240 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:29.234+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:29.476 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:29.470+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:29.650 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:29.644+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:29.914 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:29.908+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:30.081 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:30.075+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:30.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:30.192+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:30.199 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:30.195+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:30.277 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:30.270+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:30.450 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:30.443+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:30.678 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:30.674+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:30.905 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:30.899+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:31.032 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:31.029+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:31.235 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:31.229+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:31.237 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:31.232+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:31.306 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:31.300+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:31.427 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:31.421+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:31.645 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:31.642+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:31.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:31.872+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:32.003 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:31.999+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:32.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:32.197+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:32.204 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:32.201+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:32.318 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:32.311+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:32.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:32.426+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:32.669 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:32.664+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:32.835 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:32.829+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:32.972 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:32.969+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:33.218 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:33.214+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:33.219 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:33.217+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:33.332 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:33.325+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:33.467 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:33.460+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:33.627 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:33.623+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:33.832 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:33.826+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:33.935 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:33.927+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:34.205 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:34.201+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:34.207 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:34.202+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:34.335 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:34.329+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:34.474 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:34.462+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:34.664 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:34.660+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:34.775 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:34.771+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:34.875 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:34.868+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:34.980 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:34.975+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:35.187 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:35.183+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:35.188 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:35.185+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:35.354 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:35.349+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:35.504 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:35.497+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:35.707 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:35.701+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:35.798 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:35.793+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:35.863 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:35.854+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:35.998 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:35.994+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:36.159 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:36.155+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:36.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:36.160+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:36.374 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:36.368+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:36.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:36.533+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:36.730 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:36.725+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:36.815 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:36.810+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:36.873 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:36.865+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:37.004 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:37.000+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:37.128 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:37.122+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:37.134 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:37.129+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:37.410 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:37.403+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:37.588 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:37.583+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:37.748 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:37.744+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:37.809 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:37.804+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:37.902 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:37.896+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:38.041 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:38.033+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:38.115 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:38.110+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:38.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:38.112+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:38.444 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:38.438+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:38.606 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:38.600+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:38.729 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:38.725+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:38.786 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:38.781+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:38.879 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:38.873+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:39.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:39.080+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:39.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:39.163+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:39.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:39.169+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:39.401 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:39.395+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:39.656 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:39.650+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:39.749 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:39.744+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:39.808 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:39.803+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:39.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:39.873+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:40.066 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:40.055+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:40.186 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:40.181+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:40.186 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:40.183+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:40.428 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:40.422+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:40.710 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:40.703+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:40.725 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:40.719+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:40.811 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:40.806+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:40.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:40.872+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:41.078 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:41.074+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:41.218 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:41.212+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:41.219 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:41.215+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:41.413 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:41.408+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:41.699 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:41.694+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:41.770 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:41.766+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:41.854 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:41.848+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:41.857 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:41.854+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:41.879 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:41.874+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:42.122 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:42.117+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:42.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:42.224+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:42.234 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:42.230+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:42.447 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:42.441+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:42.734 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:42.726+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:42.815 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:42.811+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:42.847 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:42.840+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:42.902 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:42.894+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:42.903 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:42.897+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:43.078 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:43.072+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:43.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:43.225+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:43.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:43.228+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:43.448 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:43.441+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:43.711 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:43.703+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:43.860 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:43.852+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:43.861 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:43.855+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:43.870 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:43.864+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:43.896 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:43.890+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:44.043 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:44.039+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:44.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:44.223+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:44.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:44.227+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:44.481 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:44.473+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:44.674 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:44.666+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:44.890 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:44.882+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:44.894 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:44.888+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:44.918 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:44.913+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:44.942 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:44.934+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:45.092 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:45.087+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:45.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:45.219+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:45.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:45.225+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:45.487 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:45.480+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:45.656 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:45.648+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:45.900 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:45.889+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:45.901 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:45.892+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:45.913 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:45.906+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:45.955 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:45.951+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:46.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:46.079+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:46.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:46.225+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:46.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:46.227+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:46.466 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:46.458+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:46.619 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:46.612+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:46.887 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:46.880+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:46.907 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:46.899+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:46.908 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:46.902+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:46.938 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:46.934+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:47.133 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:47.129+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:47.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:47.262+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:47.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:47.265+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:47.478 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:47.471+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:47.586 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:47.579+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:47.889 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:47.881+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:47.890 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:47.884+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:47.916 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:47.912+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:47.937 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:47.930+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:48.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:48.154+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:48.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:48.254+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:48.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:48.256+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:48.485 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:48.479+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:48.638 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:48.631+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:48.912 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:48.905+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:48.916 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:48.911+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:48.917 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:48.911+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:48.921 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:48.917+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:49.200 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:49.196+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:49.292 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:49.287+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:49.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:49.289+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:49.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:49.439+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:49.634 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:49.627+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:49.902 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:49.897+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:49.917 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:49.909+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:49.918 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:49.912+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:49.927 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:49.920+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:50.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:50.155+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:50.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:50.319+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:50.326 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:50.321+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:50.413 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:50.405+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:50.598 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:50.591+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:50.867 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:50.863+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:50.896 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:50.889+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:50.958 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:50.948+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:50.958 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:50.953+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:51.173 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:51.168+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:51.349 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:51.343+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:51.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:51.346+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:51.414 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:51.407+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:51.600 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:51.595+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:51.838 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:51.833+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:51.881 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:51.872+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:51.915 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:51.909+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:51.919 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:51.914+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:52.191 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:52.186+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:52.348 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:52.342+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:52.348 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:52.345+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:52.397 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:52.390+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:52.628 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:52.621+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:52.811 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:52.806+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:52.883 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:52.876+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:52.907 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:52.900+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:52.908 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:52.902+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:53.208 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:53.204+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:53.328 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:53.322+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:53.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:53.325+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:53.431 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:53.424+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:53.643 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:53.636+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:53.793 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:53.788+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:53.862 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:53.855+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:53.936 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:53.928+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:53.937 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:53.931+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:54.200 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:54.196+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:54.338 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:54.332+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:54.340 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:54.336+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:54.440 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:54.432+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:54.656 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:54.648+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:54.789 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:54.785+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:54.824 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:54.817+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:54.949 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:54.942+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:54.950 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:54.944+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:55.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:55.223+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:55.330 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:55.326+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:55.333 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:55.329+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:55.485 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:55.477+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:55.684 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:55.678+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:55.797 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:55.793+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:55.874 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:55.868+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:55.913 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:55.903+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:55.913 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:55.906+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:56.274 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:56.270+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:56.358 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:56.353+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:56.360 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:56.357+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:56.502 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:56.496+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:56.657 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:56.650+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:56.829 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:56.825+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:56.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:56.868+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:56.949 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:56.943+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:56.953 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:56.948+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:57.293 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:57.288+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:57.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:57.319+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:57.326 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:57.322+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:57.532 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:57.525+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:57.628 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:57.620+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:57.817 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:57.812+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:57.901 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:57.894+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:57.953 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:57.945+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:57.954 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:57.948+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:58.295 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:58.291+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:58.317 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:58.313+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:58.319 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:58.317+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:58.570 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:58.563+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:58.581 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:58.574+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:58.790 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:58.787+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:58.920 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:58.913+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:58.933 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:58.925+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:58.934 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:58.928+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:59.257 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:48:59.253+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:48:59.372 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:59.367+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:48:59.375 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:48:59.371+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:48:59.560 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:48:59.554+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:48:59.563 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:48:59.556+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:48:59.776 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:48:59.772+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:48:59.896 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:48:59.889+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:48:59.932 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:59.924+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:48:59.933 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:48:59.928+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:48:59.989 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:48:59.984+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e326 v326) 2019-12-08T20:49:00.278 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:00.273+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:00.425 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:00.421+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:00.428 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:00.424+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:00.518 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:00.510+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:00.560 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:00.553+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:00.822 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:00.818+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:00.901 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:00.893+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:00.902 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:00.896+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:00.932 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:00.925+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:01.264 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:01.256+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:01.449 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:01.445+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:01.450 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:01.447+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:01.519 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:01.512+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:01.560 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:01.553+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:01.804 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:01.800+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:01.925 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:01.919+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:01.929 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:01.924+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:01.955 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:01.947+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:02.261 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:02.256+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:02.482 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:02.475+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:02.501 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:02.491+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:02.501 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:02.496+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:02.571 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:02.564+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:02.826 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:02.823+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:02.885 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:02.878+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:02.888 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:02.883+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:02.973 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:02.966+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:03.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:03.298+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:03.447 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:03.440+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:03.480 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:03.476+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:03.481 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:03.478+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:03.591 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:03.584+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:03.845 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:03.841+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:03.917 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:03.910+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:03.921 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:03.915+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:03.936 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:03.929+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:04.313 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:04.309+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:04.404 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:04.397+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:04.475 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:04.471+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:04.480 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:04.476+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:04.596 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:04.588+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:04.852 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:04.846+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:04.894 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:04.887+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:04.952 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:04.944+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:04.953 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:04.947+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:05.005 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:49:05.000+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e326 v326) 2019-12-08T20:49:05.354 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:05.349+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:05.407 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:05.400+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:05.505 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:05.499+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:05.508 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:05.502+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:05.629 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:05.622+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:05.896 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:05.891+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:05.903 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:05.897+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:06.006 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:05.998+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:06.007 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:06.001+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:06.317 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:06.314+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:06.382 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:06.375+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:06.539 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:06.533+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:06.541 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:06.536+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:06.631 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:06.623+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:06.914 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:06.910+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:06.940 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:06.933+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:07.034 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:07.018+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:07.035 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:07.021+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:07.310 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:07.306+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:07.349 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:07.342+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:07.495 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:07.490+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:07.496 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:07.493+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:07.675 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:07.668+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:07.953 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:07.948+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:07.986 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:07.980+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:08.025 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:08.009+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:08.025 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:08.012+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:08.333 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:08.327+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:08.387 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:08.379+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:08.548 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:08.542+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:08.549 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:08.544+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:08.669 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:08.662+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:08.956 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:08.949+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:08.997 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:08.990+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:09.017 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:09.009+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:09.018 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:09.012+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:09.317 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:09.311+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:09.357 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:09.350+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:09.535 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:09.530+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:09.538 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:09.534+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:09.699 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:09.692+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:09.925 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:09.920+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:09.975 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:09.967+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:10.023 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:49:10.014+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e326 v326) 2019-12-08T20:49:10.048 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:10.040+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:10.049 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:10.043+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:10.343 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:10.337+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:10.352 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:10.345+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:10.495 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:10.490+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:10.499 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:10.494+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:10.752 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:10.745+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:10.965 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:10.955+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:10.971 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:10.958+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:11.042 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:11.034+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:11.043 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:11.037+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:11.336 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:11.329+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:11.354 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:11.348+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:11.490 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:11.484+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:11.491 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:11.486+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:11.775 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:11.768+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:11.972 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:11.965+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:11.991 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:11.987+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:12.057 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:12.051+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:12.059 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:12.054+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:12.300 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:12.293+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:12.349 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:12.344+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:12.501 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:12.495+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:12.504 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:12.497+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:12.769 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:12.762+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:12.960 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:12.953+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:12.975 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:12.969+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:13.111 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:13.104+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:13.116 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:13.110+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:13.306 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:13.298+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:13.378 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:13.372+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:13.479 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:13.473+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:13.479 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:13.475+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:13.763 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:13.755+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:13.949 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:13.942+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:13.989 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:13.983+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:14.138 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:14.131+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:14.142 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:14.136+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:14.357 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:49:14.350+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.817383+0000 front 2019-12-08T20:47:44.769503+0000 (oldest deadline 2019-12-08T20:48:28.833228+0000) 2019-12-08T20:49:14.400 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:49:14.394+0000 1694f700 -1 osd.3 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:41.560586+0000 front 2019-12-08T20:47:41.684257+0000 (oldest deadline 2019-12-08T20:48:27.163827+0000) 2019-12-08T20:49:14.504 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:14.499+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:44.773382+0000 front 2019-12-08T20:47:44.309150+0000 (oldest deadline 2019-12-08T20:48:28.974644+0000) 2019-12-08T20:49:14.507 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:14.500+0000 1694f700 -1 osd.0 326 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:14.782 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:49:14.775+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.600546+0000 front 2019-12-08T20:47:46.452184+0000 (oldest deadline 2019-12-08T20:48:28.086235+0000) 2019-12-08T20:49:14.963 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:49:14.958+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:45.510367+0000 front 2019-12-08T20:47:45.389751+0000 (oldest deadline 2019-12-08T20:48:29.447395+0000) 2019-12-08T20:49:14.998 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:49:14.983+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:47:46.823718+0000 front 2019-12-08T20:47:46.520056+0000 (oldest deadline 2019-12-08T20:48:29.312993+0000) 2019-12-08T20:49:15.057 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:49:15.050+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 47sec e326 v326) 2019-12-08T20:49:15.165 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:15.152+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:48:01.540956+0000 (oldest deadline 2019-12-08T20:48:41.540956+0000) 2019-12-08T20:49:15.165 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:49:15.155+0000 1694f700 -1 osd.6 326 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-08T20:49:15.557 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:15.471+0000 1694f700 -1 osd.0 327 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:16.527 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:49:16.512+0000 1694f700 -1 osd.0 328 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:115340 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5448788096 gen 2] snapc 0=[] ondisk+write+known_if_redirected e322) 2019-12-08T20:49:17.742 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:49:17.737+0000 20e25700 -1 RGWWatcher::handle_error cookie 5448788096 err (107) Transport endpoint is not connected 2019-12-08T20:49:41.600 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:49:41.584+0000 30f85700 -1 osd.1 331 build_incremental_map_msg missing incremental map 331 2019-12-08T20:50:00.384 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:06:31.812 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.389 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:06:31.817 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.431 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:06:32.023 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.434 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:06:31.860 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.441 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:06:31.867 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.450 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:06:32.043 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.565 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:06:31.990 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.621 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:06:32.049 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:00.739 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:06:32.332 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:50:38.415 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:38.409+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:39.457 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:39.453+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:40.431 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:40.426+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:40.486 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:40.479+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:40.823 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:40.817+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:41.032 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:41.025+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:41.431 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:41.425+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:41.453 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:41.449+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:41.459 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:41.453+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:41.845 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:41.840+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:41.997 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:41.991+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:42.407 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:42.402+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:42.453 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:42.447+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:42.488 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:42.483+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:42.846 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:42.838+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:42.956 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:42.950+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:43.379 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:43.374+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:43.428 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:43.423+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:43.439 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:43.433+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:43.464 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:43.460+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:43.716 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:43.710+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:43.829 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:43.823+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:43.998 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:43.993+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:44.397 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:44.392+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:44.397 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:44.393+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:44.429 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:44.422+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:44.499 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:44.496+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:44.738 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:44.733+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:44.801 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:44.796+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:44.953 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:44.947+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:44.953 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:44.948+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:45.394 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:45.389+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:45.420 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:45.414+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:45.456 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:45.442+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:45.487 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:45.482+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:45.713 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:45.708+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:45.819 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:45.813+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:45.929 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:45.924+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:45.933 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:45.926+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:46.348 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:46.343+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:46.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:46.408+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:46.450 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:46.445+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:46.452 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:46.445+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:46.733 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:46.729+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:46.864 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:46.858+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:46.896 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:46.890+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:46.896 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:46.890+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:47.346 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:47.341+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:47.408 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:47.397+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:47.425 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:47.419+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:47.428 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:47.425+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:47.692 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:47.689+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:47.861 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:47.855+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:47.862 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:47.856+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:47.919 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:47.913+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:48.347 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:48.343+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:48.393 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:48.389+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:48.428 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:48.422+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:48.430 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:48.426+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:48.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:48.668+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:48.899 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:48.893+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:48.899 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:48.895+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:48.912 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:48.899+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:49.399 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:49.396+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:49.422 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:49.417+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:49.430 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:49.426+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:49.465 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:49.460+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:49.645 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:49.642+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:49.921 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:49.915+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:49.922 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:49.915+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:49.922 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:49.917+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:50.390 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:50.387+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:50.411 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:50.406+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:50.434 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:50.426+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:50.490 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:50.483+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:50.665 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:50.661+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:50.911 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:50.905+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:50.912 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:50.906+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:50.960 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:50.950+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:51.383 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:51.379+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:51.435 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:51.431+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:51.444 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:51.438+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:51.478 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:51.474+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:51.691 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:51.687+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:51.940 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:51.934+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:51.940 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:51.935+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:51.986 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:51.981+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:52.419 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:52.415+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:52.429 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:52.423+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:52.435 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:52.431+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:52.479 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:52.476+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:52.730 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:52.727+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:52.978 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:52.971+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:52.981 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:52.974+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:52.981 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:52.975+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:53.390 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:53.381+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:53.390 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:53.377+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:53.401 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:53.397+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:53.475 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:53.471+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:53.762 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:53.758+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:53.978 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:53.973+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:53.980 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:53.975+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:54.014 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:54.007+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:54.349 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:54.345+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:54.357 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:54.351+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:54.405 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:54.400+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:54.497 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:54.492+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:54.793 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:54.786+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:54.952 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:54.946+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:54.953 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:54.947+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:55.062 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:55.057+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:55.367 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:55.363+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:55.372 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:55.369+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:55.392 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:55.386+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:55.493 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:55.489+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:55.829 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:55.825+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:55.917 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:55.909+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:55.917 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:55.910+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:56.072 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:56.066+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:56.354 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:56.348+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:56.362 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:56.359+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:56.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:56.408+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:56.449 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:56.445+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:56.865 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:56.860+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:56.895 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:56.886+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:56.898 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:56.890+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:57.102 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:57.095+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:57.326 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:57.322+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:57.407 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:57.400+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:57.441 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:57.436+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:57.450 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:57.447+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:57.881 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:57.874+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:57.881 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:57.875+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:57.888 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:57.884+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:58.109 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:58.103+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:58.374 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:58.370+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:58.379 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:58.373+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:58.413 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:58.409+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:58.471 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:58.467+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:58.890 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:58.886+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:58.930 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:58.923+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:58.931 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:58.925+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:50:59.109 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:50:59.103+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:50:59.394 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:50:59.388+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:50:59.413 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:50:59.409+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:50:59.427 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:50:59.422+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:50:59.473 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:50:59.469+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:50:59.882 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:50:59.878+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:50:59.934 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:59.927+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:50:59.934 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:50:59.928+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:00.065 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:00.058+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:00.401 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:00.393+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:00.414 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:00.410+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:00.429 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:00.425+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:00.431 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:00.426+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:00.849 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:00.845+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:00.921 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:00.915+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:00.922 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:00.916+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:01.038 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:01.031+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:01.382 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:01.373+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:01.394 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:01.391+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:01.403 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:01.399+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:01.448 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:01.444+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:01.870 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:01.862+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:01.907 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:01.901+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:01.908 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:01.903+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:01.996 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:01.989+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:02.378 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:02.372+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:02.379 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:02.376+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:02.406 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:02.402+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:02.409 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:02.405+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:02.911 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:02.907+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:02.933 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:02.926+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:02.934 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:02.928+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:02.965 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:02.958+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:03.372 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:03.366+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:03.373 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:03.363+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:03.390 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:03.386+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:03.415 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:03.411+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:03.916 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:03.913+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:03.942 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:03.937+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:03.944 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:03.939+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:03.951 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:03.946+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:04.330 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:04.324+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:04.373 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:04.369+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:04.395 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:04.390+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:04.402 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:04.398+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:04.913 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:04.908+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:04.924 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:04.919+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:04.927 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:04.922+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:04.974 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:04.968+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:05.319 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:05.313+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:05.365 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:05.361+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:05.369 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:05.365+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:05.417 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:05.411+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:05.881 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:05.876+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:05.881 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:05.877+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:05.906 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:05.898+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:05.970 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:05.966+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:06.333 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:06.329+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:06.352 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:06.347+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:06.415 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:06.412+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:06.418 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:06.414+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:06.914 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:06.908+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:06.932 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:06.926+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:06.932 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:06.927+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:07.016 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:07.011+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:07.324 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:07.320+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:07.332 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:07.328+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:07.399 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:07.395+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:07.459 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:07.455+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:07.877 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:07.872+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:07.957 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:07.952+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:07.959 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:07.955+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:08.047 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:08.041+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:08.314 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:08.309+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:08.381 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:08.378+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:08.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:08.437+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:08.450 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:08.446+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:08.879 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:08.875+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:08.945 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:08.939+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:08.945 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:08.941+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:09.007 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:09.001+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:09.284 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:09.278+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:09.344 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:09.340+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:09.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:09.407+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:09.420 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:09.415+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:09.900 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:09.890+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:09.966 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:09.961+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:09.966 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:09.962+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:10.051 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:10.047+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:10.276 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:10.271+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:10.335 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:10.331+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:10.407 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:10.402+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:10.456 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:10.445+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:10.620 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:51:10.612+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 39sec e332 v332) 2019-12-08T20:51:10.918 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:10.914+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:11.004 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:10.999+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:11.006 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:11.002+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:11.020 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:11.015+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:11.290 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:11.285+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:11.346 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:11.341+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:11.413 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:11.406+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:11.425 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:11.421+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:11.965 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:11.961+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:11.974 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:11.970+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:11.975 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:11.970+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:12.005 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:12.000+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:12.304 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:12.300+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:12.334 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:12.328+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:12.405 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:12.401+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:12.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:12.407+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:12.986 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:12.981+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:12.988 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:12.984+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:13.017 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:13.013+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:13.044 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:13.020+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:13.304 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:13.298+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:13.305 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:13.301+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:13.405 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:13.401+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:13.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:13.408+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:13.961 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:13.956+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:13.964 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:13.959+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:13.998 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:13.994+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:14.021 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:14.017+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:14.272 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:14.266+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:14.279 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:14.271+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:14.369 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:14.365+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:14.387 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:14.376+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:14.958 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:14.954+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:14.961 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:14.957+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:15.011 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:15.006+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:15.056 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:15.052+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:15.247 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:15.242+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:15.321 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:15.317+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:15.339 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:15.334+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:15.341 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:15.336+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:15.636 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:51:15.633+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 39sec e332 v332) 2019-12-08T20:51:15.957 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:15.953+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:15.958 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:15.953+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:16.006 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:16.002+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:16.055 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:16.051+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:16.247 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:16.235+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:16.316 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:16.309+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:16.372 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:16.366+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:16.383 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:16.379+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:16.954 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:16.948+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:16.954 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:16.949+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:17.030 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:17.026+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:17.044 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:17.040+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:17.256 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:17.252+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:17.350 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:17.343+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:17.360 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:17.356+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:17.422 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:17.418+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:17.950 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:17.946+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:17.952 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:17.948+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:18.008 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:18.005+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:18.029 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:18.023+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:18.303 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:18.297+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:18.321 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:18.313+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:18.383 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:18.377+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:18.424 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:18.420+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:18.963 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:18.957+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:18.963 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:18.957+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:19.026 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:19.022+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:19.040 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:19.034+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:19.310 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:19.307+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:19.350 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:19.346+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:19.356 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:19.349+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:19.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:19.396+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:19.949 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:19.944+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.952783+0000 front 2019-12-08T20:49:56.526126+0000 (oldest deadline 2019-12-08T20:50:40.787395+0000) 2019-12-08T20:51:19.950 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:19.945+0000 1694f700 -1 osd.7 332 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:19.992 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:51:19.987+0000 1694f700 -1 osd.4 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:56.707607+0000 front 2019-12-08T20:49:57.393661+0000 (oldest deadline 2019-12-08T20:50:40.629330+0000) 2019-12-08T20:51:20.020 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:51:20.015+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.424656+0000 front 2019-12-08T20:50:00.873775+0000 (oldest deadline 2019-12-08T20:50:43.500359+0000) 2019-12-08T20:51:20.306 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:51:20.301+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:58.652183+0000 front 2019-12-08T20:50:00.439295+0000 (oldest deadline 2019-12-08T20:50:37.643749+0000) 2019-12-08T20:51:20.348 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:51:20.344+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:59.497695+0000 front 2019-12-08T20:50:00.937929+0000 (oldest deadline 2019-12-08T20:50:42.790654+0000) 2019-12-08T20:51:20.372 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:51:20.361+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:50:00.951507+0000 front 2019-12-08T20:49:55.988364+0000 (oldest deadline 2019-12-08T20:50:39.901958+0000) 2019-12-08T20:51:20.402 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:51:20.398+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:49:57.395421+0000 front 2019-12-08T20:49:57.400753+0000 (oldest deadline 2019-12-08T20:50:40.582238+0000) 2019-12-08T20:51:20.699 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:51:20.694+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 39sec e332 v332) 2019-12-08T20:51:21.128 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:21.119+0000 1694f700 -1 osd.7 333 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:22.378 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:51:22.311+0000 1694f700 -1 osd.7 334 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115700 2.1ds0 2:b975e364:::9161132f-da48-40b4-a853-f341a1971d48.4507.618_%e1%a0%84%e9%90%89%e0%a4%80%ef%84%91%e3%9c%83%eb%8c%8e%eb%8c%8b%e3%9c%84%e0%a4%82%eb%8c%8d%e1%a0%84%e3%9c%86%ef%84%92%ed%88%90%e5%98%86:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e332) 2019-12-08T20:51:30.915 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:08:02.342 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:30.920 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:08:02.347 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:30.928 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:08:02.354 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:30.935 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:08:02.361 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:30.936 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:08:02.529 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:30.951 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:08:02.544 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:31.018 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:08:02.446 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:31.031 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:08:02.457 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:31.087 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:08:02.680 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:51:51.621 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:51:51.607+0000 31786700 -1 osd.6 337 build_incremental_map_msg missing incremental map 337 2019-12-08T20:51:51.875 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:51:51.857+0000 34f8d700 -1 osd.6 337 build_incremental_map_msg missing incremental map 337 2019-12-08T20:51:51.876 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:51:51.860+0000 34f8d700 -1 osd.6 337 build_incremental_map_msg missing incremental map 337 2019-12-08T20:51:51.963 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:51:51.943+0000 35f8f700 -1 osd.6 337 build_incremental_map_msg missing incremental map 337 2019-12-08T20:51:53.118 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:51:53.102+0000 37792700 -1 osd.6 338 build_incremental_map_msg missing incremental map 337 2019-12-08T20:52:06.178 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:08:37.605 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.180 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:08:37.608 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.244 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:08:37.669 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.248 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:08:37.674 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.259 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:08:37.687 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.264 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:08:37.856 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.271 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:08:37.864 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.323 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:08:37.749 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:06.382 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:08:37.975 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:52:19.860 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:19.854+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:20.061 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:20.052+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:20.586 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:20.578+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:20.683 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:20.677+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:20.813 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:20.809+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:20.897 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:20.890+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:21.058 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:21.054+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:21.639 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:21.632+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:21.685 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:21.680+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:21.824 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:21.817+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:21.930 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:21.923+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:22.054 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:22.050+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:22.349 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:22.344+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:22.630 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:22.623+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:22.702 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:22.697+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:22.835 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:22.830+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:22.915 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:22.909+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:23.044 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:23.039+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:23.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:23.360+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:23.601 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:23.593+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:23.708 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:23.703+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:23.817 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:23.813+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:23.942 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:23.936+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:24.019 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:24.015+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:24.126 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:24.119+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:24.325 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:24.321+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:24.646 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:24.638+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:24.698 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:24.691+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:24.836 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:24.832+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:24.838 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:24.836+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:24.990 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:24.967+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:25.049 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:25.044+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:25.173 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:25.165+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:25.322 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:25.318+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:25.661 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:25.656+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:25.706 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:25.701+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:25.871 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:25.867+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:25.871 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:25.868+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:25.972 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:25.965+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:26.069 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:26.064+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:26.129 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:26.122+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:26.305 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:26.300+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:26.681 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:26.673+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:26.704 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:26.699+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:26.882 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:26.878+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:26.883 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:26.881+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:26.948 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:26.940+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:27.105 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:27.100+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:27.176 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:27.169+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:27.293 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:27.288+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:27.720 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:27.713+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:27.752 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:27.747+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:27.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:27.889+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:27.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:27.890+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:27.964 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:27.952+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:28.128 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:28.123+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:28.207 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:28.200+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:28.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:28.265+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:28.732 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:28.724+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:28.732 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:28.727+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:28.764 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:28.757+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:28.847 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:28.843+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:28.849 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:28.847+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:28.986 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:28.979+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:29.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:29.080+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:29.237 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:29.230+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:29.316 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:29.310+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:29.719 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:29.711+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:29.723 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:29.717+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:29.815 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:29.809+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:29.870 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:29.866+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:29.871 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:29.868+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:29.956 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:29.950+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:30.123 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:30.119+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:30.223 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:30.217+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:30.362 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:30.357+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:30.741 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:30.732+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:30.742 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:30.736+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:30.777 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:30.770+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:30.838 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:30.831+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:30.839 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:30.835+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:30.919 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:30.913+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:31.114 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:31.110+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:31.251 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:31.240+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:31.324 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:31.320+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:31.750 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:31.744+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:31.754 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:31.749+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:31.772 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:31.765+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:31.886 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:31.883+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:31.887 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:31.884+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:31.948 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:31.941+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:32.146 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:32.143+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:32.280 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:32.273+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:32.378 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:32.373+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:32.739 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:32.732+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:32.804 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:32.798+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:32.807 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:32.802+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:32.908 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:32.903+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:32.909 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:32.905+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:32.997 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:32.990+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:33.101 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:33.097+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:33.266 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:33.260+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:33.351 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:33.343+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:33.776 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:33.768+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:33.777 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:33.771+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:33.777 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:33.771+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:33.908 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:33.902+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:33.909 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:33.905+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:34.051 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:34.044+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:34.058 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:34.055+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:34.317 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:34.312+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:34.320 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:34.309+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:34.756 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:34.749+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:34.768 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:34.761+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:34.769 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:34.764+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:34.953 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:34.948+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:34.954 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:34.951+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:35.081 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:35.076+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:35.100 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:35.093+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:35.357 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:35.352+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:35.367 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:35.360+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:35.750 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:35.743+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:35.781 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:35.773+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:35.782 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:35.776+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:35.946 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:35.941+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:35.947 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:35.944+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:36.109 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:36.102+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:36.127 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:36.122+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:36.345 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:36.339+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:36.356 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:36.353+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:36.761 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:36.755+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:36.762 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:36.757+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:36.794 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:36.787+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:36.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:36.950+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:36.957 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:36.953+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:37.093 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:37.086+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:37.174 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:37.170+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:37.357 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:37.350+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:37.366 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:37.362+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:37.728 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:37.720+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:37.731 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:37.723+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:37.757 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:37.750+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:37.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:37.971+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:37.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:37.974+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:38.065 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:38.057+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:38.143 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:38.139+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:38.322 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:38.314+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:38.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:38.357+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:38.706 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:38.698+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:38.707 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:38.701+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:38.782 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:38.776+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:38.948 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:38.941+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:38.953 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:38.950+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:39.081 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:39.073+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:39.122 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:39.119+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:39.316 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:39.308+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:39.376 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:39.371+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:39.752 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:39.746+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:39.754 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:39.749+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:39.801 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:39.796+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:39.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:39.973+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:39.982 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:39.979+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:40.056 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:40.047+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:40.097 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:40.094+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:40.335 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:40.329+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:40.419 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:40.416+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:40.734 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:40.725+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:40.735 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:40.727+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:40.778 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:40.771+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:41.023 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:41.018+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:41.029 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:41.021+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:41.066 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:41.060+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:41.143 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:41.138+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:41.293 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:41.285+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:41.449 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:41.444+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:41.739 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:41.732+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:41.740 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:41.734+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:41.797 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:41.790+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:42.033 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:42.026+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:42.048 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:42.044+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:42.050 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:42.047+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:42.172 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:42.167+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:42.290 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:42.283+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:42.416 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:42.413+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:42.753 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:42.747+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:42.754 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:42.749+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:42.850 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:42.843+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:42.992 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:42.986+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:43.009 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:43.005+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:43.011 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:43.008+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:43.152 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:43.148+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:43.303 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:43.297+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:43.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:43.403+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:43.718 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:43.710+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:43.719 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:43.713+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:43.883 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:43.877+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:43.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:43.961+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:43.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:43.964+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:44.024 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:44.017+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:44.124 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:44.120+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:44.303 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:44.296+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:44.415 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:44.411+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:44.741 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:44.734+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:44.744 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:44.739+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:44.857 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:44.850+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:44.943 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:44.938+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:44.944 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:44.940+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:45.065 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:45.058+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:45.114 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:45.109+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:45.267 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:45.260+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:45.406 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:45.401+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:45.725 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:45.719+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:45.729 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:45.724+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:45.840 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:45.832+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:45.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:45.984+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:45.990 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:45.986+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:46.111 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:46.104+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:46.155 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:46.151+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:46.265 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:46.259+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:46.436 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:46.431+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:46.743 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:46.735+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:46.743 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:46.738+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:46.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:46.827+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:47.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:47.031+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:47.038 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:47.034+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:47.141 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:47.136+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:47.148 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:47.141+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:47.249 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:47.242+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:47.467 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:47.463+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:47.746 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:47.738+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:47.747 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:47.741+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:47.802 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:47.796+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:48.001 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:47.995+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:48.002 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:47.999+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:48.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:48.123+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:48.150 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:48.145+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:48.240 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:48.231+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:48.483 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:48.479+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:48.709 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:48.702+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:48.712 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:48.706+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:48.808 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:48.801+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:49.050 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:49.044+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:49.054 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:49.050+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:49.133 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:49.127+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:49.187 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:49.182+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:49.259 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:49.253+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:49.469 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:49.464+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:49.731 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:49.723+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:49.732 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:49.727+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:49.787 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:49.782+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:50.006 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:50.003+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:50.009 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:50.006+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:50.134 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:50.128+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:50.228 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:50.223+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:50.237 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:50.231+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:50.471 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:50.467+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:50.720 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:50.714+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:50.723 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:50.719+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:50.755 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:50.749+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:51.049 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:51.045+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:51.051 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:51.049+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:51.134 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:51.128+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:51.197 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:51.192+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:51.253 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:52:51.249+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e338 v338) 2019-12-08T20:52:51.263 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:51.256+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:51.511 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:51.507+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:51.745 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:51.739+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:51.751 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:51.745+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:51.755 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:51.750+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:52.055 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:52.051+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:52.056 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:52.053+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:52.137 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:52.131+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:52.177 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:52.169+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:52.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:52.257+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:52.498 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:52.495+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:52.710 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:52.705+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:52.717 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:52.711+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:52.718 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:52.714+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:53.010 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:53.006+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:53.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:53.009+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:53.145 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:53.139+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:53.155 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:53.149+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:53.309 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:53.305+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:53.499 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:53.495+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:53.697 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:53.689+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:53.744 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:53.737+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:53.745 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:53.741+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:54.027 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:54.022+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:54.029 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:54.026+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:54.134 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:54.128+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:54.209 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:54.203+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:54.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:54.298+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:54.485 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:54.481+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:54.719 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:54.713+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:54.763 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:54.756+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:54.768 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:54.761+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:55.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:55.031+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:55.043 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:55.035+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:55.142 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:55.136+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:55.264 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:55.256+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:55.314 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:55.308+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:55.512 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:52:55.508+0000 1694f700 -1 osd.2 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.328997+0000 front 2019-12-08T20:51:37.288826+0000 (oldest deadline 2019-12-08T20:52:22.168563+0000) 2019-12-08T20:52:55.735 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:52:55.729+0000 1694f700 -1 osd.5 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.915630+0000 front 2019-12-08T20:51:36.928877+0000 (oldest deadline 2019-12-08T20:52:19.786218+0000) 2019-12-08T20:52:55.781 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:55.775+0000 1694f700 -1 osd.6 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:51:47.772200+0000 (oldest deadline 2019-12-08T20:52:27.772200+0000) 2019-12-08T20:52:55.782 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:52:55.777+0000 1694f700 -1 osd.6 338 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e336) 2019-12-08T20:52:56.046 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:56.037+0000 1694f700 -1 osd.0 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.826981+0000 front 2019-12-08T20:51:37.841877+0000 (oldest deadline 2019-12-08T20:52:20.522184+0000) 2019-12-08T20:52:56.051 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:56.047+0000 1694f700 -1 osd.0 338 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:56.155 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:52:56.150+0000 1694f700 -1 osd.4 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:37.929013+0000 front 2019-12-08T20:51:38.016890+0000 (oldest deadline 2019-12-08T20:52:23.818738+0000) 2019-12-08T20:52:56.220 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:52:56.215+0000 1694f700 -1 osd.7 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:35.052454+0000 front 2019-12-08T20:51:35.050214+0000 (oldest deadline 2019-12-08T20:52:18.999918+0000) 2019-12-08T20:52:56.301 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:52:56.294+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e338 v338) 2019-12-08T20:52:56.337 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:52:56.333+0000 1694f700 -1 osd.3 338 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:51:36.065949+0000 front 2019-12-08T20:51:36.024735+0000 (oldest deadline 2019-12-08T20:52:19.488839+0000) 2019-12-08T20:52:57.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:57.122+0000 1694f700 -1 osd.0 339 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:58.154 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:52:58.148+0000 1694f700 -1 osd.0 340 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:115883 5.5 5:a93a5511:::notify.2:head [watch ping cookie 1472080656 gen 1] snapc 0=[] ondisk+write+known_if_redirected e338) 2019-12-08T20:52:59.032 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:52:59.018+0000 20e25700 -1 RGWWatcher::handle_error cookie 1472080656 err (107) Transport endpoint is not connected 2019-12-08T20:53:29.558 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.540+0000 30f85700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:53:29.645 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.618+0000 34f8d700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:53:29.717 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.687+0000 31786700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:53:29.879 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.855+0000 31786700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:53:29.938 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.914+0000 34f8d700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:53:29.952 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.917+0000 34f8d700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:53:29.993 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:53:29.977+0000 35f8f700 -1 osd.1 343 build_incremental_map_msg missing incremental map 343 2019-12-08T20:54:28.124 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:28.118+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:28.926 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:28.921+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:29.175 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:29.169+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:29.212 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:29.206+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:29.532 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:29.527+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:29.951 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:29.945+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:30.051 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:30.046+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:30.202 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:30.196+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:30.225 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:30.221+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:30.544 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:30.537+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:30.930 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:30.924+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:31.060 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:31.055+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:31.147 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:31.138+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:31.206 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:31.202+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:31.213 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:31.207+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:31.563 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:31.558+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:31.948 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:31.942+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:31.964 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:31.959+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:32.040 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:32.036+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:32.131 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:32.125+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:32.215 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:32.210+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:32.261 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:32.255+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:32.606 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:32.600+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:32.943 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:32.938+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:32.974 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:32.966+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:33.002 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:32.998+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:33.120 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:33.116+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:33.268 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:33.263+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:33.277 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:33.272+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:33.558 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:33.554+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:33.913 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:33.907+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:33.968 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:33.964+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:34.032 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:34.022+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:34.120 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:34.115+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:34.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:34.295+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:34.312 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:34.308+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:34.593 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:34.587+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:34.955 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:34.951+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:34.957 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:34.952+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:35.031 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:35.021+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:35.158 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:35.153+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:35.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:35.252+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:35.356 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:35.350+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:35.609 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:35.605+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:35.925 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:35.922+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:35.957 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:35.947+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:36.019 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:36.014+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:36.176 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:36.171+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:36.246 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:36.243+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:36.362 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:36.357+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:36.616 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:36.610+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:36.961 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:36.956+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:36.973 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:36.969+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:36.976 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:36.972+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:37.135 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:37.130+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:37.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:37.263+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:37.410 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:37.404+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:37.665 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:37.661+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:37.945 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:37.942+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:37.994 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:37.989+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:38.005 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:38.000+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:38.154 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:38.148+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:38.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:38.224+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:38.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:38.449+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:38.635 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:38.630+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:38.941 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:38.935+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:38.984 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:38.974+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:39.002 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:38.996+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:39.141 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:39.135+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:39.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:39.255+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:39.477 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:39.472+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:39.673 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:39.665+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:39.915 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:39.911+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:39.939 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:39.935+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:39.955 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:39.951+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:40.102 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:40.097+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:40.275 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:40.269+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:40.454 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:40.449+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:40.691 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:40.687+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:40.923 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:40.918+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:40.968 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:40.964+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:40.974 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:40.970+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:41.069 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:41.064+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:41.314 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:41.310+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:41.483 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:41.479+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:41.686 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:41.682+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:41.909 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:41.904+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:41.949 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:41.945+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:42.012 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:42.006+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:42.057 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:42.052+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:42.282 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:42.276+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:42.457 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:42.452+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:42.665 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:42.660+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:42.866 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:42.861+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:42.933 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:42.928+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:42.967 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:42.962+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:43.061 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:43.056+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:43.283 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:43.279+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:43.450 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:43.445+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:43.673 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:43.670+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:43.885 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:43.880+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:43.924 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:43.921+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:43.958 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:43.954+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:44.051 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:44.045+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:44.272 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:44.268+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:44.464 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:44.460+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:44.700 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:44.697+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:44.901 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:44.896+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:44.948 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:44.944+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:44.950 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:44.946+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:45.024 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:45.018+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:45.227 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:45.223+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:45.468 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:45.461+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:45.704 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:45.700+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:45.870 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:45.866+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:45.907 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:45.903+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:45.981 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:45.977+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:46.073 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:46.068+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:46.183 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:46.180+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:46.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:46.438+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:46.666 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:46.661+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:46.893 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:46.887+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:46.945 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:46.941+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:46.967 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:46.963+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:47.058 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:47.052+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:47.208 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:47.205+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:47.413 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:47.401+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:47.645 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:47.642+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:47.934 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:47.927+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:47.935 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:47.929+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:47.982 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:47.978+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:48.013 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:48.008+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:48.055 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:48.049+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:48.168 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:48.164+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:48.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:48.443+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:48.691 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:48.688+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:48.969 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:48.959+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:48.969 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:48.962+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:48.977 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:48.974+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:48.995 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:48.992+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:49.027 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:49.020+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:49.214 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:49.209+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:49.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:49.441+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:49.681 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:49.677+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:49.981 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:49.978+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:49.988 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:49.981+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:49.988 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:49.983+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:50.020 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:50.015+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:50.056 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:50.049+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:50.205 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:50.201+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:50.483 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:50.477+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:50.732 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:50.728+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:50.953 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:50.949+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:50.964 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:50.958+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:50.965 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:50.961+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:50.990 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:50.985+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:51.096 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:51.090+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:51.258 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:51.254+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:51.530 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:51.525+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:51.757 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:51.754+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:52.000 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:51.997+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:52.008 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:52.005+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:52.020 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:52.013+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:52.021 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:52.014+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:52.129 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:52.121+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:52.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:52.251+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:52.491 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:52.485+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:52.800 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:52.792+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:52.987 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:52.983+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:53.026 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:53.022+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:53.057 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:53.051+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:53.060 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:53.054+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:53.149 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:53.142+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:53.279 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:53.275+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:53.480 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:53.473+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:53.779 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:53.773+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:53.963 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:53.957+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:54.024 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:54.017+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:54.025 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:54.019+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:54.062 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:54.058+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:54.156 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:54.150+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:54.296 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:54.291+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:54.436 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:54.428+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:54.777 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:54.761+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:54.917 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:54.913+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:55.017 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:55.011+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:55.018 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:55.012+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:55.080 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:55.076+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:55.162 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:55.155+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:55.287 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:55.283+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:55.404 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:55.398+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:55.752 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:55.748+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:55.882 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:55.878+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:56.032 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:56.026+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:56.033 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:56.027+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:56.088 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:56.084+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:56.121 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:56.115+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:56.253 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:56.249+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:56.365 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:56.357+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:56.727 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:56.723+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:56.883 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:56.879+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:56.986 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:56.980+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:56.988 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:56.983+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:57.055 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:57.051+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:57.171 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:57.165+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:57.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:57.301+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:57.422 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:57.412+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:57.726 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:57.723+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:57.885 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:57.875+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:58.000 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:57.995+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:58.001 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:57.995+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:58.041 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:58.035+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:58.150 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:58.143+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:58.266 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:58.262+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:58.392 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:58.384+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:58.735 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:58.731+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:58.889 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:58.886+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:59.010 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:59.006+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:54:59.030 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:59.024+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:54:59.031 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:54:59.026+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:54:59.119 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:54:59.113+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:54:59.273 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:54:59.269+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:54:59.349 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:54:59.342+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:54:59.720 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:54:59.715+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:54:59.909 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:54:59.905+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:54:59.999 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:54:59.995+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:00.041 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:00.032+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:00.045 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:00.038+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:00.088 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:00.082+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:00.276 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:00.272+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:00.330 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:00.316+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:00.776 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:00.765+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:00.874 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:00.870+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:01.043 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:01.039+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:01.079 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:01.072+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:01.079 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:01.073+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:01.108 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:01.103+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:01.298 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:01.294+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:01.373 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:01.368+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:01.767 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:01.764+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:01.780 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:55:01.777+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 40sec e343 v343) 2019-12-08T20:55:01.902 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:01.899+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:02.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:02.067+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:02.088 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:02.082+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:02.089 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:02.083+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:02.096 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:02.090+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:02.272 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:02.260+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:02.402 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:02.396+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:02.793 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:02.789+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:02.885 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:02.881+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:03.037 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:03.031+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:03.068 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:03.060+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:03.125 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:03.119+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:03.125 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:03.120+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:03.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:03.251+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:03.412 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:03.405+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:03.762 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:03.759+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:03.906 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:03.902+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:03.994 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:03.990+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:04.060 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:04.054+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:04.172 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:04.167+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:04.172 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:04.167+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:04.298 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:04.294+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:04.460 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:04.454+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:04.756 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:04.752+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:04.922 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:04.917+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:04.978 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:04.974+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:05.098 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:05.092+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:05.171 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:05.164+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:05.171 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:05.166+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:05.343 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:05.339+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:05.447 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:05.443+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:05.798 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:05.794+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:05.916 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:05.912+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:05.977 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:05.973+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:06.143 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:55:06.137+0000 1694f700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.031210+0000 front 2019-12-08T20:53:49.503695+0000 (oldest deadline 2019-12-08T20:54:30.400130+0000) 2019-12-08T20:55:06.204 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:06.198+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:06.205 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:06.198+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:06.296 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:55:06.292+0000 1694f700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.855506+0000 front 2019-12-08T20:53:44.011954+0000 (oldest deadline 2019-12-08T20:54:28.608595+0000) 2019-12-08T20:55:06.492 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:55:06.486+0000 1694f700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.186902+0000 front 2019-12-08T20:53:48.294502+0000 (oldest deadline 2019-12-08T20:54:27.713365+0000) 2019-12-08T20:55:06.812 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:55:06.809+0000 1694f700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:45.542403+0000 front 2019-12-08T20:53:45.472022+0000 (oldest deadline 2019-12-08T20:54:29.442059+0000) 2019-12-08T20:55:06.852 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:55:06.847+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 40sec e343 v343) 2019-12-08T20:55:06.947 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:55:06.933+0000 1694f700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:46.058422+0000 front 2019-12-08T20:53:46.967718+0000 (oldest deadline 2019-12-08T20:54:29.272495+0000) 2019-12-08T20:55:07.005 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:55:07.002+0000 1694f700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:48.263516+0000 front 2019-12-08T20:53:48.264522+0000 (oldest deadline 2019-12-08T20:54:31.460345+0000) 2019-12-08T20:55:07.214 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:07.208+0000 1694f700 -1 osd.4 343 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:53:47.194366+0000 front 2019-12-08T20:53:48.871937+0000 (oldest deadline 2019-12-08T20:54:28.537664+0000) 2019-12-08T20:55:07.237 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:07.209+0000 1694f700 -1 osd.4 343 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:08.215 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:55:08.199+0000 1694f700 -1 osd.4 345 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116341 2.25s0 2:a507b1d5:::9161132f-da48-40b4-a853-f341a1971d48.4507.609_%ed%88%90%e1%a0%83%e1%a0%84%e3%9c%86%e1%a0%82%e9%90%89%e0%a4%80%e3%9c%83%ce%a9%eb%8c%8b%e0%a4%81%ed%88%8d%e9%90%89%e1%a0%82%e5%98%88:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e343) 2019-12-08T20:55:23.821 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:11:55.249 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:23.832 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:11:55.260 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:23.837 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:11:55.263 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:23.838 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:11:55.431 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:23.840 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:11:55.266 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:23.841 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:11:55.434 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:24.035 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:11:55.463 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:24.038 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:11:55.464 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:24.070 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:11:55.662 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:55:43.366 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:55:43.342+0000 34f8d700 -1 osd.6 348 build_incremental_map_msg missing incremental map 348 2019-12-08T20:55:43.393 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:55:43.384+0000 34f8d700 -1 osd.6 348 build_incremental_map_msg missing incremental map 348 2019-12-08T20:55:43.446 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:55:43.408+0000 3578e700 -1 osd.6 348 build_incremental_map_msg missing incremental map 348 2019-12-08T20:55:43.594 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:55:43.576+0000 3578e700 -1 osd.6 348 build_incremental_map_msg missing incremental map 348 2019-12-08T20:55:43.777 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:55:43.764+0000 3478c700 -1 osd.6 348 build_incremental_map_msg missing incremental map 348 2019-12-08T20:55:45.953 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:55:45.917+0000 37792700 -1 osd.6 349 build_incremental_map_msg missing incremental map 348 2019-12-08T20:56:04.221 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:04.217+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:05.208 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:05.203+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:05.569 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:05.562+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:05.741 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:05.735+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:06.189 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:06.185+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:06.282 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:06.252+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:06.546 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:06.539+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:06.718 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:06.710+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:07.211 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:07.207+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:07.304 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:07.297+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:07.583 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:07.575+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:07.618 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:07.611+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:07.667 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:07.663+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:08.242 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:08.237+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:08.354 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:08.347+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:08.546 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:08.541+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:08.660 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:08.653+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:08.661 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:08.658+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:09.252 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:09.248+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:09.346 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:09.339+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:09.603 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:09.593+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:09.643 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:09.638+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:09.708 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:09.703+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:10.230 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:10.225+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:10.311 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:10.304+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:10.575 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:10.568+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:10.614 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:10.606+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:10.685 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:10.681+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:11.275 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:11.270+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:11.278 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:11.271+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:11.578 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:11.571+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:11.612 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:11.604+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:11.708 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:11.705+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:11.711 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:11.708+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:12.291 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:12.288+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:12.314 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:12.307+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:12.567 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:12.560+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:12.607 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:12.600+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:12.709 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:12.704+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:12.711 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:12.708+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:13.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:13.249+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:13.297 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:13.290+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:13.538 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:13.531+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:13.609 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:13.601+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:13.683 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:13.679+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:13.683 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:13.680+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:14.290 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:14.287+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:14.340 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:14.333+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:14.362 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:14.357+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:14.497 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:14.488+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:14.525 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:14.517+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:14.589 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:14.581+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:14.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:14.701+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:14.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:14.703+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:15.264 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:15.260+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:15.366 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:15.358+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:15.389 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:15.384+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:15.522 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:15.513+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:15.544 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:15.541+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:15.564 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:15.551+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:15.659 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:15.655+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:15.660 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:15.657+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:16.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:16.227+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:16.359 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:16.353+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:16.437 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:16.432+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:16.534 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:16.530+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:16.554 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:16.548+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:16.570 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:16.563+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:16.676 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:16.672+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:16.683 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:16.676+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:17.243 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:17.240+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:17.396 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:17.392+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:17.409 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:17.403+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:17.545 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:17.540+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:17.596 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:17.590+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:17.615 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:17.609+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:17.709 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:17.704+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:17.710 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:17.706+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:18.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:18.205+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:18.376 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:18.371+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:18.441 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:18.437+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:18.501 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:18.497+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:18.580 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:18.575+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:18.645 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:18.632+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:18.721 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:18.710+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:18.721 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:18.712+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:19.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:19.250+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:19.374 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:19.368+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:19.403 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:19.396+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:19.404 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:19.399+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:19.527 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:19.523+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:19.627 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:19.620+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:19.638 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:19.633+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:19.716 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:19.713+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:19.718 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:19.716+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:20.235 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:20.230+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:20.393 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:20.387+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:20.447 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:20.440+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:20.448 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:20.443+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:20.551 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:20.547+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:20.659 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:20.654+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:20.688 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:20.682+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:20.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:20.690+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:20.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:20.693+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:21.261 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:21.257+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:21.387 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:21.381+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:21.482 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:21.474+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:21.484 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:21.478+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:21.528 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:21.524+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:21.675 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:21.669+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:21.683 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:21.678+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:21.717 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:21.712+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:21.718 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:21.714+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:22.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:22.228+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:22.391 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:22.385+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:22.482 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:22.475+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:22.483 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:22.478+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:22.527 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:22.522+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:22.678 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:22.671+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:22.679 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:22.675+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:22.692 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:22.686+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:22.719 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:22.711+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:23.200 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:23.195+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:23.440 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:23.435+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:23.535 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:23.529+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:23.536 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:23.532+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:23.570 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:23.566+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:23.646 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:23.640+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:23.691 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:23.687+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:23.691 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:23.688+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:23.752 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:23.746+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:24.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:24.250+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:24.414 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:24.403+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:24.568 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:24.563+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:24.569 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:24.565+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:24.601 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:24.596+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:24.602 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:24.597+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:24.703 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:24.699+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:24.704 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:24.701+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:24.733 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:24.726+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:25.218 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:25.213+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:25.403 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:25.396+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:25.557 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:25.554+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:25.598 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:25.593+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:25.602 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:25.598+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:25.643 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:25.637+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:25.743 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:25.737+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:25.743 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:25.740+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:25.771 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:25.766+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:26.211 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:26.207+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:26.441 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:26.434+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:26.587 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:26.583+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:26.635 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:26.630+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:26.646 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:26.634+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:26.688 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:26.682+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:26.750 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:26.742+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:26.765 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:26.761+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:26.768 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:26.765+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:27.216 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:27.212+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:27.416 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:27.410+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:27.624 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:27.620+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:27.666 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:27.659+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:27.667 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:27.662+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:27.688 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:27.682+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:27.742 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:27.736+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:27.777 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:27.773+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:27.778 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:27.775+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:28.195 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:28.190+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:28.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:28.446+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:28.613 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:28.608+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:28.712 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:28.705+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:28.713 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:28.708+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:28.722 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:28.717+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:28.728 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:28.722+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:28.770 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:28.765+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:28.772 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:28.769+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:29.163 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:29.159+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:29.496 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:29.491+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:29.624 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:29.620+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:29.680 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:29.672+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:29.680 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:29.675+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:29.712 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:29.706+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:29.738 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:29.732+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:29.773 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:29.770+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:29.774 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:29.771+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:30.126 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:30.121+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:30.504 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:30.495+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:30.661 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:30.654+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:30.664 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:30.658+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:30.669 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:30.663+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:30.691 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:30.684+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:30.727 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:30.723+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:30.728 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:30.721+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:30.729 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:30.726+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:31.163 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:31.159+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:31.546 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:31.532+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:31.628 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:31.622+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:31.630 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:31.623+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:31.646 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:31.640+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:31.666 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:31.662+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:31.709 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:31.702+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:31.748 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:31.745+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:31.749 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:31.746+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:32.119 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:32.116+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:32.535 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:32.527+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:32.595 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:32.588+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:32.599 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:32.594+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:32.618 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:32.612+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:32.673 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:32.666+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:32.715 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:32.710+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:32.758 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:32.754+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:32.760 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:32.758+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:33.130 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:33.126+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:33.512 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:33.505+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:33.608 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:33.597+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:33.610 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:33.600+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:33.610 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:33.600+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:33.721 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:33.714+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:33.730 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:33.726+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:33.805 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:33.802+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:33.806 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:33.803+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:34.139 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:34.136+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:34.474 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:34.468+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:34.566 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:34.558+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:34.567 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:34.561+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:34.593 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:34.586+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:34.712 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:34.705+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:34.737 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:34.733+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:34.824 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:34.819+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:34.827 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:34.824+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:35.095 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:35.092+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:35.521 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:35.514+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:35.548 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:35.541+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:35.552 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:35.546+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:35.586 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:35.580+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:35.707 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:35.702+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:35.747 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:35.739+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:35.823 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:35.819+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:35.825 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:35.822+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:36.123 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:36.120+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:36.509 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:36.503+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:36.512 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:36.507+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:36.558 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:36.552+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:36.615 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:36.607+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:36.726 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:36.720+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:36.729 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:36.722+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:36.822 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:36.815+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:36.822 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:36.819+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:37.067 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:56:37.065+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e349 v349) 2019-12-08T20:56:37.100 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:37.088+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:37.466 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:37.460+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:37.471 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:37.464+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:37.526 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:37.519+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:37.657 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:37.651+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:37.682 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:37.678+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:37.767 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:37.760+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:37.788 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:37.783+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:37.789 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:37.785+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:38.116 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:38.111+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:38.523 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:38.515+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:38.524 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:38.519+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:38.567 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:38.559+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:38.666 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:38.658+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:38.699 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:38.694+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:38.787 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:38.783+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:38.790 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:38.787+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:38.794 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:38.787+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:39.145 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:39.141+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:39.507 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:39.499+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:39.508 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:39.502+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:39.591 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:39.584+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:39.691 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:39.683+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:39.694 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:39.690+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:39.749 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:39.745+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:39.751 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:39.749+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:39.825 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:39.818+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:40.108 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:40.102+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:40.533 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:40.527+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:40.534 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:40.529+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:40.586 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:40.580+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:40.649 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:40.642+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:40.703 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:40.699+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:40.715 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:40.710+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:40.716 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:40.712+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:40.832 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:40.825+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:41.107 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:41.102+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:41.582 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:41.576+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:41.585 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:41.580+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:41.597 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:41.591+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:41.635 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:41.624+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:41.676 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:41.673+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:41.677 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:41.674+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:41.700 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:41.696+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:41.859 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:41.852+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:42.080 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:56:42.076+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e349 v349) 2019-12-08T20:56:42.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:42.136+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:42.635 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:42.627+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:42.636 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:42.630+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:42.639 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:42.635+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:42.640 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:42.637+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:42.646 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:42.634+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:42.670 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:42.662+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:42.725 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:42.720+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:42.821 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:42.810+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:43.170 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:43.165+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:43.624 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:43.616+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:43.625 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:43.619+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:43.639 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:43.635+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:43.642 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:43.639+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:43.645 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:43.637+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:43.669 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:43.662+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:43.766 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:43.763+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:43.835 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:43.828+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:44.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:44.143+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:44.610 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:44.603+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:44.613 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:44.608+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:44.641 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:44.634+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:44.674 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:44.668+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:44.676 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:44.670+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:44.677 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:44.671+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:44.750 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:44.745+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:44.854 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:44.847+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:45.104 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:45.099+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:45.645 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:45.638+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:45.648 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:45.643+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:45.649 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:45.645+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:45.657 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:45.649+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:45.658 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:45.652+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:45.695 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:45.688+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:45.757 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:45.753+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:45.865 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:45.855+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:46.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:46.077+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:46.617 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:46.610+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:46.618 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:46.613+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:46.632 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:46.626+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:46.636 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:46.632+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:46.655 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:46.649+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:46.724 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:46.711+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:46.793 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:46.788+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:46.893 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:46.885+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:47.093 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:56:47.088+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e349 v349) 2019-12-08T20:56:47.114 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:47.108+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:47.611 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:47.600+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:47.619 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:47.610+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:47.636 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:47.630+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:47.638 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:47.633+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:47.661 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:47.656+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:47.686 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:47.681+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:47.750 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:47.744+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:47.936 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:47.929+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:48.147 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:48.141+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:48.597 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:48.589+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:48.598 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:48.592+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:48.665 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:48.657+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:48.678 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:48.672+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:48.681 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:48.677+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:48.700 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:48.694+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:48.721 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:48.714+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:48.896 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:48.890+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:49.141 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:49.135+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:49.569 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:49.559+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:49.569 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:49.562+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:49.673 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:49.668+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:49.674 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:49.667+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:49.676 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:49.671+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:49.712 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:49.706+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:49.754 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:49.746+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:49.938 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:49.932+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:50.145 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:50.140+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:50.560 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:50.552+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:50.560 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:50.555+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:50.679 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:50.672+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:50.679 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:50.672+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:50.680 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:50.675+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:50.703 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:50.698+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:50.803 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:50.796+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:50.955 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:50.948+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:51.166 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:51.161+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:51.564 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:51.555+0000 1694f700 -1 osd.6 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:55:39.156378+0000 (oldest deadline 2019-12-08T20:56:19.156378+0000) 2019-12-08T20:56:51.566 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:51.559+0000 1694f700 -1 osd.6 349 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:51.638 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:51.633+0000 1694f700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.953837+0000 front 2019-12-08T20:55:23.839417+0000 (oldest deadline 2019-12-08T20:56:05.306819+0000) 2019-12-08T20:56:51.639 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:51.635+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:51.658 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:56:51.653+0000 1694f700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:23.268895+0000 front 2019-12-08T20:55:23.280402+0000 (oldest deadline 2019-12-08T20:56:13.636093+0000) 2019-12-08T20:56:51.705 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:56:51.698+0000 1694f700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:20.068033+0000 front 2019-12-08T20:55:19.960362+0000 (oldest deadline 2019-12-08T20:56:05.229925+0000) 2019-12-08T20:56:51.818 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:56:51.811+0000 1694f700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.720722+0000 front 2019-12-08T20:55:21.574120+0000 (oldest deadline 2019-12-08T20:56:05.656671+0000) 2019-12-08T20:56:51.926 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:56:51.919+0000 1694f700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.321679+0000 front 2019-12-08T20:55:21.448316+0000 (oldest deadline 2019-12-08T20:56:07.193844+0000) 2019-12-08T20:56:52.133 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:56:52.126+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e349 v349) 2019-12-08T20:56:52.155 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:56:52.152+0000 1694f700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:55:21.315447+0000 front 2019-12-08T20:55:21.309749+0000 (oldest deadline 2019-12-08T20:56:04.117588+0000) 2019-12-08T20:56:52.712 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:52.633+0000 1694f700 -1 osd.6 350 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e347) 2019-12-08T20:56:52.735 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T20:56:52.672+0000 3378a700 -1 osd.6 350 build_incremental_map_msg missing incremental map 348 2019-12-08T20:56:53.007 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:52.974+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:54.135 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:56:54.108+0000 1694f700 -1 osd.0 351 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116509 5.5 5:a93a5511:::notify.2:head [notify cookie 363947648 in=260b] snapc 0=[] ondisk+read+known_if_redirected e347) 2019-12-08T20:56:54.390 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T20:56:54.371+0000 20e25700 -1 RGWWatcher::handle_error cookie 5439797808 err (107) Transport endpoint is not connected 2019-12-08T20:58:16.580 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:16.575+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:16.799 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:16.794+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:16.934 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:16.930+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:17.181 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:17.175+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:17.557 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:17.552+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:17.788 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:17.783+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:17.908 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:17.903+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:18.164 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:18.158+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:18.369 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:18.365+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:18.599 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:18.594+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:18.612 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:18.606+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:18.780 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:18.771+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:18.926 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:18.921+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:19.111 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:19.104+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:19.206 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:19.200+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:19.326 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:19.322+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:19.565 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:19.560+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:19.592 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:19.586+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:19.793 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:19.789+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:19.904 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:19.899+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:20.155 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:20.148+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:20.198 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:20.193+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:20.319 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:20.314+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:20.527 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:20.520+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:20.588 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:20.581+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:20.753 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:20.749+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:20.911 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:20.907+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:21.146 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:21.140+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:21.176 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:21.170+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:21.320 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:21.315+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:21.510 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:21.504+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:21.621 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:21.615+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:21.757 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:21.752+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:21.906 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:21.900+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:22.192 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:22.186+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:22.213 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:22.207+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:22.294 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:22.288+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:22.525 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:22.521+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:22.598 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:22.593+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:22.792 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:22.787+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:22.904 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:22.900+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:23.242 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:23.237+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:23.243 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:23.237+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:23.347 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:23.343+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:23.577 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:23.573+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:23.580 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:23.575+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:23.818 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:23.815+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:23.866 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:23.863+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:24.293 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:24.279+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:24.294 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:24.285+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:24.368 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:24.365+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:24.582 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:24.577+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:24.608 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:24.602+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:24.824 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:24.821+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:24.845 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:24.841+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:25.245 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:25.239+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:25.292 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:25.287+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:25.344 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:25.340+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:25.608 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:25.605+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:25.620 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:25.613+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:25.800 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:25.796+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:25.860 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:25.856+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:26.218 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:26.213+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:26.324 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:26.320+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:26.339 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:26.333+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:26.610 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:26.603+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:26.629 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:26.621+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:26.789 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:26.785+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:26.819 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:26.816+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:27.172 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:27.166+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:27.286 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:27.282+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:27.377 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:27.367+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:27.609 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:27.601+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:27.640 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:27.634+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:27.692 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:14:59.120 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.709 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:14:59.137 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.728 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:14:59.321 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.732 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:14:59.325 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.754 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:14:59.180 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.767 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:14:59.194 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.802 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:27.799+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:27.821 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:27.817+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:27.933 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:14:59.525 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:27.963 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:14:59.389 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:28.015 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:14:59.443 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:58:28.188 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:28.183+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:28.275 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:28.271+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:28.365 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:28.359+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:28.592 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:28.586+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:28.682 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:28.679+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:28.790 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:28.786+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:28.873 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:28.869+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:29.167 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:29.162+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:29.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:29.258+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:29.405 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:29.400+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:29.554 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:29.547+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:29.654 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:29.650+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:29.834 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:29.830+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:29.835 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:29.830+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:30.171 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:30.165+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:30.278 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:30.274+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:30.369 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:30.362+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:30.550 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:30.544+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:30.625 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:30.621+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:30.794 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:30.786+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:30.850 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:30.844+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:31.134 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:31.128+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:31.233 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:31.229+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:31.399 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:31.393+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:31.542 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:31.536+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:31.609 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:31.605+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:31.782 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:31.778+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:31.825 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:31.821+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:32.160 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:32.155+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:32.218 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:32.213+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:32.445 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:32.439+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:32.522 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:32.516+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:32.607 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:32.603+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:32.761 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:32.757+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:32.825 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:32.821+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:33.121 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:33.115+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:33.269 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:33.264+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:33.436 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:33.430+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:33.558 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:33.553+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:33.600 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:33.596+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:33.766 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:33.762+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:33.859 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:33.856+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:34.122 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:34.115+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:34.318 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:34.313+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:34.482 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:34.475+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:34.538 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:34.532+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:34.638 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:34.634+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:34.808 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:34.804+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:34.906 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:34.903+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:35.143 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:35.137+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:35.325 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:35.322+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:35.530 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:35.525+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:35.533 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:35.527+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:35.677 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:35.674+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:35.830 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:35.826+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:35.888 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:35.885+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:36.105 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:36.098+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:36.300 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:36.297+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:36.546 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:36.540+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:36.569 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:36.563+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:36.687 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:36.670+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:36.837 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:36.823+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:36.927 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:36.923+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:37.074 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:37.068+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:37.266 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:37.261+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:37.536 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:37.530+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:37.547 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:37.542+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:37.641 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:37.638+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:37.835 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:37.831+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:37.919 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:37.914+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:38.087 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:38.081+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:38.299 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:38.295+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:38.551 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:38.545+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:38.567 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:38.561+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:38.656 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:38.652+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:38.878 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:38.874+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:38.881 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:38.878+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:39.079 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:39.074+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:39.306 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:39.302+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:39.554 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:39.542+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:39.612 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:39.609+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:39.614 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:39.608+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:39.880 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:39.877+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:39.904 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:39.901+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:40.038 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:40.031+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:40.280 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:40.276+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:40.554 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:40.549+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:40.605 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:40.601+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:40.627 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:40.623+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:40.879 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:40.876+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:40.897 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:40.893+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:41.055 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:41.050+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:41.253 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:41.249+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:41.526 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:41.520+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:41.629 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:41.624+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:41.650 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:41.646+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:41.908 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:41.904+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:41.924 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:41.920+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:42.073 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:42.065+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:42.286 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:42.282+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:42.532 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:42.526+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:42.616 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:42.611+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:42.617 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:42.611+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:42.876 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:42.873+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:42.941 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:42.927+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:43.112 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:43.104+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:43.255 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:43.250+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:43.542 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:43.537+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:43.665 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:43.661+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:43.669 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:43.663+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:43.910 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:43.907+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:43.937 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:43.934+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:44.160 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:44.154+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:44.272 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:44.268+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:44.507 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:44.501+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:44.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:44.691+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:44.700 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:44.689+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:44.900 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:44.896+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:44.908 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:44.903+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:45.192 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:45.186+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:45.293 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:45.290+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:45.463 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:45.458+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:45.707 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:45.702+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:45.708 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:45.699+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:45.869 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:45.866+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:45.920 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:45.917+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:46.181 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:46.175+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:46.306 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:46.303+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:46.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:46.440+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:46.679 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:46.673+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:46.681 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:46.676+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:46.906 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:46.902+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:46.934 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:46.929+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:47.153 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:47.148+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:47.282 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:47.278+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:47.494 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:47.489+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:47.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:47.702+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:47.728 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:47.722+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:47.950 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:47.947+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:47.958 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:47.955+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:48.179 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:48.173+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:48.293 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:48.290+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:48.508 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:48.503+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:48.696 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:48.690+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:48.739 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:48.736+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:48.921 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:48.917+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:48.993 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:48.989+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:49.186 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:49.180+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:49.251 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:49.247+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:49.558 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:49.552+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:49.698 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:49.692+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:49.768 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:49.764+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:49.940 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:49.937+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:49.953 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:49.950+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:50.172 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:50.166+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:50.233 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:50.229+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:50.570 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:50.564+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:50.751 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:50.745+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:50.771 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:50.767+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:50.914 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:50.910+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:51.005 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:51.001+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:51.173 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:51.167+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:51.245 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:51.241+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:51.561 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:51.555+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:51.747 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:51.744+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:51.775 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:51.770+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:51.875 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:51.871+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:58:52.021 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:58:52.017+0000 1694f700 -1 osd.2 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:30.381449+0000 front 2019-12-08T20:57:30.208769+0000 (oldest deadline 2019-12-08T20:58:16.071243+0000) 2019-12-08T20:58:52.145 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:58:52.139+0000 1694f700 -1 osd.5 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:34.163929+0000 front 2019-12-08T20:57:34.475348+0000 (oldest deadline 2019-12-08T20:58:17.042411+0000) 2019-12-08T20:58:52.240 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:58:52.236+0000 1694f700 -1 osd.3 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494500+0000 front 2019-12-08T20:57:35.492390+0000 (oldest deadline 2019-12-08T20:58:17.369707+0000) 2019-12-08T20:58:52.601 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:58:52.597+0000 1694f700 -1 osd.7 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:32.769599+0000 front 2019-12-08T20:57:33.885549+0000 (oldest deadline 2019-12-08T20:58:18.195055+0000) 2019-12-08T20:58:52.710 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T20:58:52.706+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e354 v354) 2019-12-08T20:58:52.724 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:58:52.721+0000 1694f700 -1 osd.0 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.357445+0000 front 2019-12-08T20:57:35.469604+0000 (oldest deadline 2019-12-08T20:58:16.209439+0000) 2019-12-08T20:58:52.755 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:58:52.751+0000 1694f700 -1 osd.4 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.552509+0000 front 2019-12-08T20:57:35.552952+0000 (oldest deadline 2019-12-08T20:58:18.577447+0000) 2019-12-08T20:58:52.843 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T20:58:52.838+0000 1694f700 -1 osd.1 354 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T20:57:35.494270+0000 front 2019-12-08T20:57:33.863021+0000 (oldest deadline 2019-12-08T20:58:16.376726+0000) 2019-12-08T20:59:22.967 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:15:54.395 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:22.974 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:15:54.402 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:22.981 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:15:54.407 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:22.986 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:15:54.412 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:22.990 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:15:54.583 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:23.004 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:15:54.597 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:23.045 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:15:54.472 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:23.091 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:15:54.517 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:23.096 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:15:54.688 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T20:59:43.030 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:43.028+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:44.053 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:44.051+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:45.037 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:45.035+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:46.064 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:46.043+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:47.018 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:47.013+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:47.992 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:47.988+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:48.959 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:48.955+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:49.937 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:49.934+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:50.001 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:49.995+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:50.903 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:50.895+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:50.905 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:50.899+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:50.980 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:50.974+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:51.043 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:51.036+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:51.061 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:51.056+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:51.666 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:51.660+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:51.950 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:51.944+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:51.952 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:51.948+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:52.012 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:52.006+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:52.031 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:52.024+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:52.059 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:52.048+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:52.638 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:52.633+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:52.940 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:52.935+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:52.941 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:52.938+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:53.012 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:53.006+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:53.035 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:53.028+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:53.037 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:53.032+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:53.101 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:53.091+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:53.678 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:53.673+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:53.941 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:53.936+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:53.941 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:53.937+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:54.045 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:54.040+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:54.053 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:54.045+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:54.057 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:54.052+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:54.129 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:54.119+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:54.706 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:54.695+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:54.898 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:54.892+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:54.899 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:54.895+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:55.040 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:55.035+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:55.062 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:55.056+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:55.079 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:55.073+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:55.137 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:55.131+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:55.736 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:55.730+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:55.892 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:55.889+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:55.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:55.891+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:56.030 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:56.026+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:56.039 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:56.033+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:56.086 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:56.080+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:56.138 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:56.132+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:56.758 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:56.754+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:56.906 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:56.902+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:56.907 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:56.904+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:57.046 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:57.041+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:57.052 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:57.045+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:57.139 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:57.133+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:57.142 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:57.136+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:57.788 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:57.783+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:57.951 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:57.947+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:57.953 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:57.951+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:58.034 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:58.027+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:58.054 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:58.050+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:58.099 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:58.092+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:58.114 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:58.107+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:58.749 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:58.744+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:58.945 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:58.940+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:58.945 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:58.943+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T20:59:59.047 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T20:59:59.041+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T20:59:59.060 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T20:59:59.056+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T20:59:59.137 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T20:59:59.129+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T20:59:59.137 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T20:59:59.129+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T20:59:59.758 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T20:59:59.754+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T20:59:59.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:59.964+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T20:59:59.968 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T20:59:59.965+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:00.065 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:00.059+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:00.094 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:00.087+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:00.100 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:00.095+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:00.155 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:00.148+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:00.757 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:00.753+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:00.906 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:00.901+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:00.996 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:00.991+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:00.999 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:00.996+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:01.050 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:01.043+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:01.081 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:01.073+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:01.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:01.079+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:01.163 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:01.156+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:01.775 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:01.771+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:01.951 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:01.946+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:01.966 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:01.961+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:01.967 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:01.963+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:02.035 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:02.027+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:02.104 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:02.100+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:02.105 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:02.098+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:02.204 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:02.197+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:02.819 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:02.815+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:02.942 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:02.936+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:02.945 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:02.943+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:03.002 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:02.997+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:03.080 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:03.077+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:03.087 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:03.078+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:03.147 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:03.139+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:03.236 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:03.229+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:03.790 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:03.786+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:03.976 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:03.971+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:03.978 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:03.974+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:04.041 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:04.035+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:04.043 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:04.038+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:04.070 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:04.064+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:04.111 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:04.104+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:04.263 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:04.255+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:04.767 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:04.761+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:05.006 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:05.001+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:05.009 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:05.006+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:05.014 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:05.010+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:05.029 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:05.024+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:05.088 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:05.082+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:05.154 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:05.147+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:05.285 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:05.280+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:05.775 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:05.771+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:05.995 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:05.989+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:06.000 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:05.997+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:06.055 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:06.052+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:06.056 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:06.053+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:06.081 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:06.075+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:06.206 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:06.199+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:06.297 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:06.290+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:06.818 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:06.815+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:06.997 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:06.991+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:07.021 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:07.017+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:07.052 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:07.048+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:07.053 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:07.050+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:07.066 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:07.059+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:07.236 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:07.228+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:07.324 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:07.317+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:07.825 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:07.822+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:07.985 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:07.980+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:08.021 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:08.017+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:08.083 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:08.080+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:08.085 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:08.082+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:08.110 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:08.104+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:08.281 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:08.275+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:08.327 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:08.321+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:08.825 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:08.820+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:08.945 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:08.937+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:08.946 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:08.940+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:09.013 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:09.008+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:09.110 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:09.105+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:09.111 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:09.104+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:09.111 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:09.107+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:09.283 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:09.276+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:09.340 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:09.333+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:09.874 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:09.871+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:09.985 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:09.978+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:09.986 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:09.981+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:10.023 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:10.020+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:10.090 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:10.086+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:10.091 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:10.087+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:10.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:10.123+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:10.330 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:10.323+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:10.372 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:10.366+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:10.876 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:10.872+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:10.948 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:10.941+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:10.951 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:10.946+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:11.022 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:11.017+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:11.095 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:11.090+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:11.095 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:11.092+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:11.147 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:11.140+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:11.308 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:11.301+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:11.357 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:11.350+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:11.912 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:11.909+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:11.926 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:11.919+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:11.928 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:11.922+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:12.058 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:12.044+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:12.059 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:12.052+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:12.059 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:12.054+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:12.133 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:12.126+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:12.340 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:12.331+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:12.362 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:12.355+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:12.875 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:12.868+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:12.899 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:12.883+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:12.907 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:12.902+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:13.002 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:12.998+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:13.055 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:13.052+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:13.056 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:13.053+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:13.182 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:13.175+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:13.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:13.306+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:13.403 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:13.397+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:13.911 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:13.907+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:13.929 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:13.921+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:13.930 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:13.924+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:13.994 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:13.987+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:14.020 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:14.016+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:14.023 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:14.020+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:14.209 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:14.203+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:14.325 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:14.319+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:14.449 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:14.442+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:14.897 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:14.894+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:14.927 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:14.921+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:14.930 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:14.926+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:14.993 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:14.984+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:15.057 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:15.051+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:15.058 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:15.054+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:15.214 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:15.207+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:15.301 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:15.290+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:15.493 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:15.486+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:15.859 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:15.854+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:15.959 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:15.952+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:15.960 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:15.954+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:15.973 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:15.969+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:16.033 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:16.027+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:16.035 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:16.031+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:16.228 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:16.220+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:16.272 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:16.266+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:16.505 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:16.490+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:16.908 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:16.899+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:16.966 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:16.962+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:16.984 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:16.978+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:16.988 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:16.983+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:17.021 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:17.016+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:17.022 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:17.018+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:17.246 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:17.236+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:17.275 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:17.268+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:17.465 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:17.457+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:17.889 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:17.884+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:17.979 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:17.975+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:17.980 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:17.977+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:17.987 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:17.984+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:17.995 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:17.988+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:17.999 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:17.994+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:18.202 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:18.196+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:18.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:18.225+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:18.478 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:18.470+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:18.853 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:18.848+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:18.954 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:18.947+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:18.982 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:18.979+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:18.986 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:18.982+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:19.023 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:19.017+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:19.027 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:19.022+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:19.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:19.225+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:19.248 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:19.241+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:19.515 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:19.505+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:19.826 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:19.823+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:19.923 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:19.918+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:19.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:19.984+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:19.989 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:19.986+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:20.017 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:20.008+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:20.027 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:20.011+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:20.233 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:20.219+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:20.255 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:20.248+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:20.499 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:20.492+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:20.788 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:20.781+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:20.964 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:20.959+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:20.995 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:20.989+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:20.999 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:20.994+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:21.030 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:21.024+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:21.032 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:21.027+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:21.198 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:21.191+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:21.215 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:21.208+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:21.519 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:21.514+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:21.771 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:21.766+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:21.924 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:21.921+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:21.962 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:21.955+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:21.962 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:21.957+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:22.027 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:22.022+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:22.028 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:22.025+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:22.230 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:22.222+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:22.243 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:22.236+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:22.550 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:22.544+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:22.817 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:22.813+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:22.891 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:22.887+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:23.011 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:23.004+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:23.012 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:23.008+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:23.012 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:23.007+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:23.013 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:23.010+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:23.041 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:00:23.035+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e363 v363) 2019-12-08T21:00:23.209 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:23.203+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:23.268 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:23.262+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:23.534 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:23.527+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:23.838 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:23.834+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:23.927 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:23.922+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:23.985 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:23.978+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:23.986 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:23.980+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:24.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:24.035+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:24.041 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:24.037+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:24.183 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:24.176+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:24.238 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:24.231+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:24.540 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:24.532+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:24.884 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:24.879+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:24.951 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:24.946+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:25.000 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:24.992+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:25.001 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:24.995+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:25.015 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:25.010+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:25.018 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:25.012+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:25.218 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:25.210+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:25.262 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:25.248+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:25.570 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:25.563+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:25.866 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:25.862+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:25.938 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:25.934+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:25.963 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:25.956+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:25.965 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:25.959+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:26.051 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:26.046+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:26.054 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:26.051+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:26.234 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:26.228+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:26.252 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:26.245+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:26.605 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:26.597+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:26.849 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:26.844+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:26.930 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:26.922+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:26.931 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:26.925+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:26.954 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:26.950+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:27.092 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:27.088+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:27.093 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:27.090+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:27.253 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:27.247+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:27.261 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:27.255+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:27.608 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:27.601+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:27.860 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:27.856+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:27.928 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:27.925+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:27.949 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:27.943+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:27.954 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:27.948+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:28.051 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:00:28.048+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e363 v363) 2019-12-08T21:00:28.138 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:28.135+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:28.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:28.138+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:28.233 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:28.226+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:28.300 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:28.293+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:28.646 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:28.639+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:28.913 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:28.909+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:28.928 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:28.922+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:28.964 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:28.954+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:28.965 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:28.959+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:29.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:29.144+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:29.152 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:29.148+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:29.272 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:29.265+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:29.273 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:29.266+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:29.676 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:29.669+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:29.928 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:29.923+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:29.961 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:29.949+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:29.978 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:29.969+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:29.978 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:29.972+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:30.134 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:30.126+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:30.134 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:30.129+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:30.244 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:30.238+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:30.307 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:30.301+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:30.693 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:30.685+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:30.908 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:30.904+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:30.977 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:30.969+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:30.979 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:30.972+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:30.985 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:30.981+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:31.096 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:31.084+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:31.097 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:31.087+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:31.253 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:31.246+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:31.293 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:31.282+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:31.663 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:31.657+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:31.950 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:31.940+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:31.950 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:31.945+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:31.952 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:31.948+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:31.980 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:31.977+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:32.133 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:32.128+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:32.134 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:32.130+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:32.264 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:32.257+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:32.286 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:32.279+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:32.664 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:32.656+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:32.928 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:32.922+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:32.950 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:32.943+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:32.951 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:32.946+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:32.992 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:32.988+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:33.066 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:00:33.060+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e363 v363) 2019-12-08T21:00:33.108 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:33.103+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:33.109 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:33.105+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:33.285 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:33.278+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:33.317 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:33.304+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:33.669 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:33.653+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:33.897 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:33.892+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:33.937 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:33.930+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:33.938 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:33.932+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:33.956 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:33.950+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:34.105 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:34.099+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:34.106 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:34.102+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:34.289 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:34.281+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:34.299 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:34.293+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:34.702 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:34.696+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:34.891 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:34.885+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:34.900 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:34.891+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:34.901 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:34.895+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:34.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:34.948+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:35.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:35.147+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:35.154 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:35.150+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:35.278 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:35.269+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:35.311 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:35.305+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:35.702 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:35.696+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:35.866 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:35.857+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:35.868 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:35.860+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:35.914 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:35.908+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:35.929 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:35.925+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:36.171 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:36.166+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:36.172 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:36.167+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:36.289 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:36.284+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:36.314 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:36.310+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:36.675 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:36.667+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:36.863 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:36.855+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:36.866 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:36.859+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:36.911 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:36.906+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:36.944 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:36.937+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:37.158 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:37.153+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:37.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:37.156+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:37.300 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:00:37.293+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:08.456969+0000 front 2019-12-08T20:59:08.723536+0000 (oldest deadline 2019-12-08T20:59:50.463460+0000) 2019-12-08T21:00:37.302 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:37.297+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:37.650 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:00:37.643+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.787707+0000 front 2019-12-08T20:59:05.997893+0000 (oldest deadline 2019-12-08T20:59:49.848587+0000) 2019-12-08T21:00:37.842 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:37.834+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T20:59:28.231601+0000 (oldest deadline 2019-12-08T21:00:08.231601+0000) 2019-12-08T21:00:37.844 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:00:37.838+0000 1694f700 -1 osd.6 363 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:109680 7.d 7.25f726ad (undecoded) ondisk+retry+write+known_if_redirected e358) 2019-12-08T21:00:37.922 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:00:37.918+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:05.486759+0000 front 2019-12-08T20:59:05.590975+0000 (oldest deadline 2019-12-08T20:59:51.384821+0000) 2019-12-08T21:00:37.957 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:00:37.948+0000 1694f700 -1 osd.3 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:10.002615+0000 front 2019-12-08T20:59:09.618828+0000 (oldest deadline 2019-12-08T20:59:52.426661+0000) 2019-12-08T21:00:38.100 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:00:38.095+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 41sec e363 v363) 2019-12-08T21:00:38.193 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:38.187+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:09.810136+0000 front 2019-12-08T20:59:09.811202+0000 (oldest deadline 2019-12-08T20:59:50.593343+0000) 2019-12-08T21:00:38.194 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:38.189+0000 1694f700 -1 osd.0 363 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:38.267 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:00:38.262+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T20:59:07.716030+0000 front 2019-12-08T20:59:07.510534+0000 (oldest deadline 2019-12-08T20:59:50.119110+0000) 2019-12-08T21:00:39.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:00:39.167+0000 1694f700 -1 osd.0 364 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:116943 5.5 5:a93a5511:::notify.2:head [watch ping cookie 384182768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e356) 2019-12-08T21:00:40.784 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:00:40.709+0000 20e25700 -1 RGWWatcher::handle_error cookie 384182768 err (107) Transport endpoint is not connected 2019-12-08T21:01:06.805 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:01:06.796+0000 124b0700 -1 osd.5 367 build_incremental_map_msg missing incremental map 367 2019-12-08T21:01:06.892 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:01:06.860+0000 3578e700 -1 osd.5 367 build_incremental_map_msg missing incremental map 367 2019-12-08T21:01:06.928 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:01:06.865+0000 32788700 -1 osd.5 367 build_incremental_map_msg missing incremental map 367 2019-12-08T21:02:08.557 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:08.551+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:08.564 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:08.560+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:09.023 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:09.016+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:09.246 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:09.240+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:09.473 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:09.467+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:09.556 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:09.551+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:09.567 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:09.561+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:09.986 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:09.980+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:10.258 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:10.252+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:10.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:10.444+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:10.524 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:10.518+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:10.578 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:10.572+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:10.604 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:10.598+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:10.973 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:10.967+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:11.239 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:11.235+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:11.424 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:11.418+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:11.486 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:11.481+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:11.578 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:11.573+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:11.625 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:11.619+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:11.978 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:11.971+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:12.258 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:12.255+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:12.411 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:12.405+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:12.458 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:12.454+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:12.620 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:12.613+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:12.628 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:12.624+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:12.936 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:12.930+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:13.245 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:13.242+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:13.276 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:13.272+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:13.449 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:13.443+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:13.508 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:13.502+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:13.600 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:13.594+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:13.600 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:13.596+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:13.948 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:13.942+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:14.260 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:14.255+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:14.289 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:14.285+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:14.461 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:14.454+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:14.560 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:14.557+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:14.586 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:14.582+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:14.597 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:14.592+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:14.959 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:14.953+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:15.260 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:15.257+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:15.304 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:15.300+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:15.459 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:15.452+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:15.550 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:15.547+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:15.562 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:15.558+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:15.610 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:15.604+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:15.974 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:15.969+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:16.311 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:16.307+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:16.332 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:16.328+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:16.510 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:16.504+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:16.562 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:16.559+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:16.617 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:16.603+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:16.623 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:16.618+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:17.025 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:17.014+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:17.302 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:17.299+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:17.312 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:17.308+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:17.536 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:17.532+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:17.561 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:17.555+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:17.591 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:17.587+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:17.611 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:17.605+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:18.039 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:18.034+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:18.264 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:18.260+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:18.300 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:18.296+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:18.512 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:18.509+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:18.548 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:18.542+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:18.620 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:18.616+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:18.641 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:18.634+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:19.061 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:19.056+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:19.282 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:19.277+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:19.299 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:19.296+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:19.479 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:19.472+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:19.520 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:19.515+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:19.576 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:19.573+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:19.680 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:19.675+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:20.098 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:20.091+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:20.302 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:20.297+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:20.329 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:20.325+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:20.487 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:20.481+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:20.509 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:20.504+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:20.582 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:20.578+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:20.695 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:20.691+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:21.136 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:21.132+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:21.280 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:21.275+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:21.355 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:21.352+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:21.522 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:21.517+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:21.533 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:21.525+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:21.620 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:21.616+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:21.658 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:21.653+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:22.171 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:22.165+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:22.279 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:22.275+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:22.385 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:22.364+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:22.514 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:22.510+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:22.524 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:22.518+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:22.622 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:22.618+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:22.636 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:22.631+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:23.145 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:23.140+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:23.322 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:23.318+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:23.406 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:23.401+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:23.503 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:23.499+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:23.508 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:23.504+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:23.618 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:23.615+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:23.621 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:23.616+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:24.123 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:24.118+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:24.320 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:24.316+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:24.420 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:24.417+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:24.505 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:24.498+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:24.546 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:24.543+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:24.573 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:24.570+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:24.585 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:24.580+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:25.147 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:25.142+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:25.365 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:25.361+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:25.376 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:25.373+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:25.521 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:25.517+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:25.551 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:25.546+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:25.553 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:25.548+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:25.623 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:25.620+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:26.203 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:26.198+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:26.349 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:26.345+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:26.393 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:26.388+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:26.505 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:26.501+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:26.540 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:26.535+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:26.585 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:26.580+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:26.643 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:26.640+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:27.206 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:27.201+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:27.305 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:27.301+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:27.348 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:27.344+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:27.477 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:27.473+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:27.536 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:27.532+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:27.614 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:27.609+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:27.690 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:27.685+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:28.250 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:28.245+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:28.340 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:28.336+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:28.344 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:28.341+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:28.503 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:28.498+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:28.506 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:28.502+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:28.645 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:28.641+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:28.668 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:28.665+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:29.267 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:29.263+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:29.304 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:29.301+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:29.334 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:29.331+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:29.504 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:29.499+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:29.550 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:29.547+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:29.622 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:29.616+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:29.640 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:29.636+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:30.267 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:30.262+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:30.313 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:30.309+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:30.343 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:30.339+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:30.513 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:30.508+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:30.540 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:30.536+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:30.611 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:30.608+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:30.655 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:30.650+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:31.313 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:31.307+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:31.329 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:31.323+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:31.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:31.360+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:31.499 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:31.494+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:31.557 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:31.553+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:31.617 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:31.612+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:31.631 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:31.628+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:32.310 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:32.305+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:32.327 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:32.324+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:32.330 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:32.326+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:32.496 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:32.491+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:32.547 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:32.543+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:32.617 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:32.604+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:32.652 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:32.648+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:33.308 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:33.299+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:33.323 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:33.319+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:33.379 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:33.375+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:33.509 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:33.504+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:33.573 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:33.567+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:33.610 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:33.603+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:33.618 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:33.615+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:34.282 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:34.278+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:34.342 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:34.336+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:34.432 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:34.428+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:34.558 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:34.553+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:34.571 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:34.565+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:34.595 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:34.589+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:34.638 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:34.634+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:35.289 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:35.285+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:35.300 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:35.295+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:35.406 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:35.402+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:35.520 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:35.515+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:35.533 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:35.527+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:35.565 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:35.559+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:35.687 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:35.683+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:36.274 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:36.271+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:36.275 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:36.267+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:36.394 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:36.390+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:36.479 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:36.474+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:36.528 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:36.523+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:36.532 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:36.525+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:36.695 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:36.691+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:37.279 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:37.272+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:37.325 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:37.322+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:37.353 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:37.349+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:37.513 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:37.508+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:37.527 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:37.520+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:37.555 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:37.549+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:37.697 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:37.690+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:38.313 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:38.307+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:38.341 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:38.337+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:38.377 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:38.371+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:38.508 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:38.503+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:38.528 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:38.525+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:38.546 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:38.541+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:38.679 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:38.675+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:39.314 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:39.308+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:39.361 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:39.355+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:39.407 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:39.402+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:39.468 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:39.462+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:39.509 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:39.503+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:39.529 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:39.525+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:39.663 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:39.660+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:40.319 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:40.315+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:40.329 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:40.323+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:40.404 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:40.400+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:40.493 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:40.487+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:40.501 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:40.495+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:40.532 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:40.528+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:40.629 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:40.626+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:41.338 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:41.335+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:41.353 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:41.347+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:41.387 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:41.367+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:41.483 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:41.477+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:41.511 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:41.505+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:41.528 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:41.525+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:41.618 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:41.614+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:42.325 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:42.313+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:42.365 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:42.361+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:42.411 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:42.408+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:42.444 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:42.439+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:42.491 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:42.485+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:42.496 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:42.493+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:42.615 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:42.611+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:02:43.279 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:02:43.273+0000 1694f700 -1 osd.4 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.836855+0000 front 2019-12-08T21:01:25.377063+0000 (oldest deadline 2019-12-08T21:02:08.091211+0000) 2019-12-08T21:02:43.402 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:02:43.396+0000 1694f700 -1 osd.5 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:26.433484+0000 front 2019-12-08T21:01:27.145806+0000 (oldest deadline 2019-12-08T21:02:08.131067+0000) 2019-12-08T21:02:43.406 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:02:43.397+0000 1694f700 -1 osd.2 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.597015+0000 front 2019-12-08T21:01:27.175349+0000 (oldest deadline 2019-12-08T21:02:08.807255+0000) 2019-12-08T21:02:43.437 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:02:43.432+0000 1694f700 -1 osd.1 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:28.633980+0000 front 2019-12-08T21:01:27.186361+0000 (oldest deadline 2019-12-08T21:02:12.442447+0000) 2019-12-08T21:02:43.511 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:02:43.505+0000 1694f700 -1 osd.7 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:25.058228+0000 front 2019-12-08T21:01:25.055879+0000 (oldest deadline 2019-12-08T21:02:08.744666+0000) 2019-12-08T21:02:43.528 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:02:43.524+0000 1694f700 -1 osd.0 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.921556+0000 front 2019-12-08T21:01:23.907657+0000 (oldest deadline 2019-12-08T21:02:09.762402+0000) 2019-12-08T21:02:43.529 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:02:43.525+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 43sec e368 v368) 2019-12-08T21:02:43.615 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:02:43.607+0000 1694f700 -1 osd.3 368 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:01:23.935596+0000 front 2019-12-08T21:01:23.852621+0000 (oldest deadline 2019-12-08T21:02:07.721940+0000) 2019-12-08T21:03:08.737 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:19:40.165 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:08.748 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:19:40.175 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:08.756 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:19:40.349 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:08.758 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:19:40.184 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:08.761 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:19:40.188 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:08.768 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:19:40.361 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:09.009 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:19:40.435 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:09.012 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:19:40.440 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:09.018 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:19:40.611 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:03:21.599 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:21.517+0000 32f89700 -1 osd.6 373 build_incremental_map_msg missing incremental map 373 2019-12-08T21:03:21.701 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:21.542+0000 35f8f700 -1 osd.6 373 build_incremental_map_msg missing incremental map 373 2019-12-08T21:03:21.719 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:21.575+0000 32f89700 -1 osd.6 373 build_incremental_map_msg missing incremental map 373 2019-12-08T21:03:21.722 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:21.589+0000 32f89700 -1 osd.6 373 build_incremental_map_msg missing incremental map 373 2019-12-08T21:03:22.438 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:22.426+0000 37792700 -1 osd.6 373 build_incremental_map_msg missing incremental map 373 2019-12-08T21:03:22.481 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:22.462+0000 3378a700 -1 osd.6 373 build_incremental_map_msg missing incremental map 373 2019-12-08T21:03:23.252 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:23.118+0000 11caf700 -1 osd.6 374 build_incremental_map_msg missing incremental map 374 2019-12-08T21:03:23.314 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:23.143+0000 35f8f700 -1 osd.6 374 build_incremental_map_msg missing incremental map 374 2019-12-08T21:03:23.490 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:23.292+0000 32788700 -1 osd.6 374 build_incremental_map_msg missing incremental map 374 2019-12-08T21:03:36.743 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:36.739+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:37.742 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:37.737+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:38.699 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:38.695+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:39.721 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:39.716+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:40.707 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:40.703+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:41.756 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:41.751+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:41.975 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:41.970+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:42.120 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:42.113+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:42.712 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:42.709+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:42.713 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:42.711+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:42.961 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:42.956+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:43.073 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:43.064+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:43.079 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:43.072+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:43.688 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:43.683+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:43.690 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:43.685+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:43.983 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:43.977+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:44.080 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:44.073+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:44.125 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:44.117+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:44.130 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:44.123+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:44.131 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:44.124+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:44.692 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:44.687+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:44.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:44.690+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:44.983 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:44.978+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:45.042 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:45.038+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:45.085 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:45.079+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:45.096 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:45.089+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:45.098 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:45.091+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:45.729 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:45.725+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:45.730 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:45.727+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:45.982 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:45.977+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:46.000 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:45.997+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:46.060 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:46.053+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:46.068 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:46.060+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:46.088 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:46.080+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:46.712 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:46.708+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:46.714 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:46.712+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:46.981 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:46.978+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:46.988 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:46.981+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:47.044 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:47.036+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:47.109 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:47.099+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:47.117 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:47.107+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:47.717 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:47.713+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:47.719 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:47.714+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:47.972 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:47.967+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:48.014 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:48.008+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:48.075 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:48.069+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:48.110 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:48.104+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:48.138 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:48.131+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:48.684 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:48.680+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:48.685 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:48.681+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:48.967 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:48.961+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:49.059 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:49.054+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:49.075 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:49.069+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:49.113 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:49.106+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:49.127 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:49.117+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:49.647 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:49.641+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:49.648 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:49.644+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:49.948 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:49.944+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:50.048 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:50.044+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:50.083 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:50.077+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:50.125 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:50.120+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:50.141 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:50.122+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:50.669 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:50.663+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:50.669 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:50.666+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:50.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:50.946+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:51.025 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:51.021+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:51.079 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:51.071+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:51.109 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:51.102+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:51.112 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:51.104+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:51.671 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:51.668+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:51.673 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:51.671+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:51.994 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:51.990+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:52.003 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:52.000+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:52.105 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:52.099+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:52.127 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:52.122+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:52.129 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:52.123+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:52.643 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:52.638+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:52.644 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:52.640+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:52.965 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:52.962+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:53.031 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:53.027+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:53.070 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:53.064+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:53.137 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:53.129+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:53.171 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:53.164+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:53.437 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:53.430+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:53.608 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:53.604+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:53.610 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:53.607+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:53.928 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:53.924+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:54.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:54.032+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:54.048 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:54.042+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:54.140 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:54.133+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:54.172 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:54.165+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:54.485 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:54.480+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:54.629 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:54.624+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:54.631 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:54.628+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:54.918 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:54.913+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:55.014 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:55.011+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:55.099 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:55.088+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:55.120 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:55.113+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:55.172 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:55.164+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:55.514 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:55.507+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:55.608 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:55.604+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:55.612 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:55.608+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:55.946 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:55.942+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:56.029 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:56.025+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:56.088 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:56.081+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:56.101 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:56.095+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:56.211 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:56.205+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:56.505 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:56.499+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:56.654 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:56.649+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:56.657 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:56.654+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:56.906 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:56.901+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:57.050 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:57.043+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:57.078 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:57.074+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:57.110 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:57.103+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:57.235 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:57.227+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:57.472 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:57.466+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:03:57.478 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:57.472+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:57.613 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:57.607+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:57.614 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:57.610+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:57.935 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:57.931+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:58.077 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:58.073+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:58.097 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:58.088+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:58.163 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:58.156+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:58.254 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:58.242+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:58.487 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:58.479+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:03:58.489 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:58.483+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:58.594 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:58.590+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:58.597 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:58.594+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:58.901 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:58.898+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:03:59.108 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:03:59.105+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:03:59.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:03:59.123+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:03:59.146 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:03:59.139+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:03:59.236 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:03:59.230+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:03:59.520 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:59.513+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:03:59.525 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:03:59.520+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:03:59.563 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:59.559+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:03:59.564 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:03:59.560+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:03:59.884 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:03:59.880+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:00.122 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:00.119+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:00.123 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:00.117+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:00.137 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:00.130+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:00.267 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:00.260+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:00.526 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:00.517+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:00.528 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:00.522+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:00.595 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:00.590+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:00.595 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:00.592+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:00.898 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:00.894+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:01.083 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:01.077+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:01.106 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:01.100+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:01.125 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:01.120+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:01.316 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:01.311+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:01.505 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:01.493+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:01.508 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:01.498+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:01.575 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:01.569+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:01.576 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:01.572+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:01.906 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:01.901+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:02.077 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:02.069+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:02.092 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:02.087+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:02.180 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:02.172+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:02.360 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:02.354+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:02.506 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:02.496+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:02.517 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:02.501+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:02.563 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:02.552+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:02.570 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:02.558+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:02.907 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:02.903+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:03.093 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:03.085+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:03.104 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:03.100+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:03.221 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:03.214+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:03.352 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:03.345+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:03.551 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:03.543+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:03.555 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:03.547+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:03.603 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:03.599+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:03.603 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:03.600+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:03.900 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:03.896+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:04.094 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:04.088+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:04.133 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:04.129+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:04.196 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:04.190+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:04.329 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:04.321+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:04.545 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:04.538+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:04.555 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:04.541+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:04.625 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:04.619+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:04.627 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:04.623+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:04.946 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:04.943+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:05.112 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:05.105+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:05.125 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:05.115+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:05.215 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:05.209+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:05.395 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:05.389+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:05.533 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:05.528+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:05.538 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:05.534+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:05.589 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:05.585+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:05.590 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:05.588+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:05.979 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:05.975+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:06.113 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:06.108+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:06.115 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:06.108+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:06.247 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:06.243+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:06.437 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:06.432+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:06.564 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:06.557+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:06.566 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:06.561+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:06.625 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:06.620+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:06.625 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:06.622+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:06.947 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:06.942+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:07.105 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:07.101+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:07.117 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:07.112+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:07.237 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:07.231+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:07.450 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:07.444+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:07.600 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:07.592+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:07.601 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:07.596+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:07.608 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:07.603+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:07.609 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:07.606+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:07.933 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:07.928+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:08.076 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:08.072+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:08.130 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:08.126+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:08.242 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:08.237+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:08.457 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:08.450+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:08.570 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:08.567+0000 1694f700 -1 osd.0 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.897733+0000 front 2019-12-08T21:02:56.862597+0000 (oldest deadline 2019-12-08T21:03:41.802448+0000) 2019-12-08T21:04:08.572 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:08.569+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:08.633 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:08.628+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:08.640 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:08.633+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:08.895 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:08.889+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:09.047 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:09.043+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:09.185 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:09.177+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:09.285 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:09.266+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:09.461 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:09.456+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:09.576 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:09.573+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:09.615 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:09.607+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:09.617 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:09.611+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:09.873 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:09.865+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:10.012 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:10.008+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:10.189 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:10.182+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:10.281 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:10.274+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:10.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:10.426+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:10.568 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:10.565+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:10.571 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:10.564+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:10.573 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:10.567+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:10.865 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:10.861+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:11.026 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:11.023+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:11.240 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:11.234+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:11.319 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:11.314+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:11.392 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:11.386+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:11.576 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:11.568+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:11.588 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:11.572+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:11.627 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:11.621+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:11.826 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:11.816+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:12.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:12.031+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:12.256 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:12.250+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:12.362 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:12.357+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:12.372 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:12.365+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:12.549 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:12.543+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:12.554 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:12.549+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:12.677 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:12.672+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:12.795 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:12.791+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:13.082 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:13.077+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:13.235 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:13.229+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:13.361 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:13.355+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:13.408 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:13.403+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:13.541 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:13.532+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:13.542 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:13.536+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:13.659 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:13.656+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:13.761 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:04:13.755+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e374 v374) 2019-12-08T21:04:13.814 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:13.810+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:14.054 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:14.050+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:14.260 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:14.247+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:14.388 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:14.382+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:14.401 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:14.395+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:14.557 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:14.550+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:14.559 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:14.553+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:14.618 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:14.614+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:14.824 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:14.820+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:15.028 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:15.024+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:15.209 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:15.204+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:15.397 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:15.391+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:15.444 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:15.438+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:15.524 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:15.516+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:15.526 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:15.520+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:15.625 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:15.621+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:15.784 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:15.779+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:16.030 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:16.027+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:16.225 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:16.215+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:16.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:16.379+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:16.424 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:16.418+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:16.518 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:16.509+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:16.520 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:16.513+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:16.651 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:16.648+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:16.816 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:16.813+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:17.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:17.031+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:17.267 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:17.260+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:17.396 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:17.389+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:17.433 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:17.425+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:17.499 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:17.492+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:17.505 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:17.499+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:17.686 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:17.683+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:17.807 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:17.803+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:18.041 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:18.037+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:18.247 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:18.240+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:18.421 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:18.414+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:18.486 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:18.476+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:18.496 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:18.486+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:18.498 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:18.490+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:18.709 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:18.705+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:18.762 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:18.757+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:18.775 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:04:18.770+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e374 v374) 2019-12-08T21:04:19.015 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:19.012+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:19.211 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:19.204+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:19.417 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:19.410+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:19.451 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:19.443+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:19.453 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:19.447+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:19.459 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:19.453+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:19.736 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:19.733+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:19.750 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:19.747+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:20.017 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:20.013+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:20.263 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:20.256+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:20.391 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:20.385+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:20.418 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:20.411+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:20.437 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:20.431+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:20.443 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:20.437+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:20.728 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:20.723+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:20.802 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:20.798+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:20.981 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:20.976+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:21.268 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:21.262+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:21.389 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:21.383+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:21.446 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:21.438+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:21.448 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:21.442+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:21.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:21.449+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:21.758 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:21.753+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:21.772 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:21.768+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:21.939 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:21.934+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:22.229 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:22.219+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:22.388 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:22.381+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:22.408 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:22.400+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:22.411 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:22.404+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:22.501 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:22.493+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:22.727 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:22.722+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:22.731 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:22.726+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:22.941 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:22.936+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:23.206 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:23.199+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:23.363 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:23.355+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:23.413 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:23.404+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:23.422 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:23.408+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:23.516 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:23.509+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:23.757 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:23.752+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:23.777 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:23.773+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:23.784 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:04:23.780+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e374 v374) 2019-12-08T21:04:23.972 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:23.964+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:24.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:24.224+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:24.367 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:24.360+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:24.452 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:24.446+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:24.463 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:24.451+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:24.568 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:24.561+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:24.737 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:24.733+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:24.805 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:24.801+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:24.986 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:24.981+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:25.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:25.225+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:25.404 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:25.398+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:25.480 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:25.471+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:25.482 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:25.476+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:25.573 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:25.566+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:25.753 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:25.748+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:25.788 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:25.783+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:25.991 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:25.986+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:26.212 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:26.205+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:26.451 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:26.439+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:26.454 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:26.448+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:26.455 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:26.447+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:26.531 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:26.524+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:26.728 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:26.723+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:26.809 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:26.803+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:27.031 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:27.025+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:27.215 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:27.207+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:27.462 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:27.456+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:27.497 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:27.490+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:27.498 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:27.489+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:27.500 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:27.494+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:27.738 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:27.734+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:27.827 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:27.822+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:28.058 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:28.054+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:28.207 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:28.199+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:28.506 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:04:28.499+0000 1694f700 -1 osd.5 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:57.534111+0000 front 2019-12-08T21:02:57.239472+0000 (oldest deadline 2019-12-08T21:03:41.313247+0000) 2019-12-08T21:04:28.539 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:28.532+0000 1694f700 -1 osd.6 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 ever on either front or back, first ping sent 2019-12-08T21:03:16.862711+0000 (oldest deadline 2019-12-08T21:03:56.862711+0000) 2019-12-08T21:04:28.545 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:28.539+0000 1694f700 -1 osd.6 374 get_health_metrics reporting 17 slow ops, oldest is osd_op(client.4507.0:126360 7.d 7.27d2557d (undecoded) ondisk+retry+read+known_if_redirected e372) 2019-12-08T21:04:28.547 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:04:28.541+0000 1694f700 -1 osd.4 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:58.603426+0000 front 2019-12-08T21:02:58.661105+0000 (oldest deadline 2019-12-08T21:03:43.628431+0000) 2019-12-08T21:04:28.774 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:28.768+0000 1694f700 -1 osd.0 374 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:28.809 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:04:28.805+0000 1694f700 -1 osd.3 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:59.712267+0000 front 2019-12-08T21:02:59.084412+0000 (oldest deadline 2019-12-08T21:03:43.131600+0000) 2019-12-08T21:04:29.048 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:04:29.042+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 42sec e374 v374) 2019-12-08T21:04:29.100 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:04:29.094+0000 1694f700 -1 osd.2 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:02:56.410395+0000 front 2019-12-08T21:02:56.515506+0000 (oldest deadline 2019-12-08T21:03:41.057064+0000) 2019-12-08T21:04:29.241 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:04:29.235+0000 1694f700 -1 osd.7 374 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:03:00.478736+0000 front 2019-12-08T21:03:00.477378+0000 (oldest deadline 2019-12-08T21:03:42.559648+0000) 2019-12-08T21:04:29.264 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:21:00.691 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.277 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:21:00.705 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.298 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:21:00.891 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.300 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:21:00.893 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.308 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:21:00.734 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.347 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:21:00.773 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.429 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:21:01.022 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.516 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:21:00.942 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.605 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:21:01.031 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:04:29.620 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:04:29.446+0000 37792700 -1 osd.6 375 build_incremental_map_msg missing incremental map 374 2019-12-08T21:04:30.086 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:04:30.036+0000 1694f700 -1 osd.0 375 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:117570 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5432396224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e370) 2019-12-08T21:04:31.634 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:04:31.609+0000 20e25700 -1 RGWWatcher::handle_error cookie 5432396224 err (107) Transport endpoint is not connected 2019-12-08T21:04:56.093 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:04:56.007+0000 34f8d700 -1 osd.1 379 build_incremental_map_msg missing incremental map 379 2019-12-08T21:04:56.286 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:04:56.092+0000 3578e700 -1 osd.1 379 build_incremental_map_msg missing incremental map 379 2019-12-08T21:04:56.344 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:04:56.153+0000 31786700 -1 osd.1 379 build_incremental_map_msg missing incremental map 379 2019-12-08T21:04:56.352 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:04:56.164+0000 3578e700 -1 osd.1 379 build_incremental_map_msg missing incremental map 379 2019-12-08T21:04:57.041 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:04:56.884+0000 37792700 -1 osd.1 379 build_incremental_map_msg missing incremental map 379 2019-12-08T21:05:50.877 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:50.873+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:51.882 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:51.878+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:52.935 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:52.931+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:53.044 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:53.039+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:53.546 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:53.540+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:53.931 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:53.927+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:54.081 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:54.072+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:54.178 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:05:54.173+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:05:54.540 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:54.535+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:54.570 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:05:54.565+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:05:54.953 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:54.948+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:05:54.953 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:54.949+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:55.125 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:55.120+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:55.218 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:05:55.213+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:05:55.523 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:55.518+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:55.544 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:05:55.539+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:05:55.970 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:55.965+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:05:55.972 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:55.969+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:56.119 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:05:56.114+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:05:56.126 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:56.121+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:56.242 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:05:56.237+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:05:56.522 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:05:56.517+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:05:56.558 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:56.553+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:57.002 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:56.997+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:05:57.003 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:56.998+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:57.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:57.144+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:57.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:05:57.156+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:05:57.239 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:05:57.234+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:05:57.476 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:05:57.471+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:05:57.580 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:57.575+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:57.977 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:05:57.973+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:05:58.007 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:58.003+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:05:58.010 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:58.006+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:58.121 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:58.115+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:58.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:05:58.159+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:05:58.196 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:05:58.191+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:05:58.521 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:05:58.517+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:05:58.585 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:58.580+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:59.009 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:59.004+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:05:59.012 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:59.008+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:05:59.013 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:05:59.008+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:05:59.131 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:05:59.127+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:05:59.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:05:59.204+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:05:59.217 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:05:59.212+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:05:59.505 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:05:59.500+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:05:59.596 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:05:59.590+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:05:59.998 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:59.993+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:00.000 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:05:59.995+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:00.009 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:00.002+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:00.181 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:00.176+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:00.234 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:00.229+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:00.241 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:00.234+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:00.499 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:00.496+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:00.567 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:00.564+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:01.001 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:00.995+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:01.011 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:01.006+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:01.012 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:01.007+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:01.199 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:01.192+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:01.222 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:01.218+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:01.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:01.223+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:01.521 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:01.517+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:01.608 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:01.604+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:02.020 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:02.014+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:02.043 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:02.036+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:02.044 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:02.038+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:02.177 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:02.169+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:02.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:02.199+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:02.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:02.207+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:02.478 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:02.474+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:02.655 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:02.651+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:02.987 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:02.980+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:03.068 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:03.060+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:03.069 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:03.062+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:03.202 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:03.197+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:03.212 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:03.208+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:03.221 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:03.215+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:03.439 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:03.430+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:03.701 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:03.697+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:04.031 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:04.025+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:04.033 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:04.027+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:04.033 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:04.028+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:04.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:04.193+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:04.252 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:04.246+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:04.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:04.254+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:04.419 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:04.415+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:04.742 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:04.738+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:05.037 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:05.032+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:05.081 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:05.075+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:05.084 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:05.076+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:05.166 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:05.162+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:05.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:05.228+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:05.242 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:05.237+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:05.453 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:05.449+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:05.725 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:05.719+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:06.025 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:06.019+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:06.127 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:06.120+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:06.127 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:06.122+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:06.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:06.145+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:06.221 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:06.215+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:06.258 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:06.255+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:06.423 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:06.419+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:06.761 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:06.753+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:07.062 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:07.055+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:07.110 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:07.103+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:07.110 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:07.104+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:07.135 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:07.131+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:07.207 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:07.201+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:07.217 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:07.213+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:07.423 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:07.418+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:07.779 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:07.776+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:08.085 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:08.080+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:08.103 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:08.097+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:08.108 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:08.103+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:08.108 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:08.104+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:08.167 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:08.162+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:08.205 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:08.197+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:08.436 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:08.432+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:08.753 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:08.749+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:09.082 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:09.077+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:09.085 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:09.080+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:09.106 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:09.100+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:09.129 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:09.123+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:09.135 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:09.132+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:09.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:09.169+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:09.449 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:09.445+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:09.748 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:09.743+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:10.123 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:10.115+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:10.123 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:10.117+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:10.126 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:10.119+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:10.129 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:10.123+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:10.136 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:10.132+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:10.162 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:10.155+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:10.454 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:10.450+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:10.744 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:10.740+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:11.129 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:11.120+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:11.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:11.133+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:11.150 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:11.145+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:11.153 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:11.148+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:11.167 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:11.163+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:11.173 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:11.167+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:11.477 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:11.472+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:11.715 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:11.710+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:12.098 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:12.093+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:12.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:12.101+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:12.138 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:12.131+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:12.146 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:12.132+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:12.183 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:12.177+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:12.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:12.212+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:12.514 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:12.510+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:12.729 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:12.725+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:13.134 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:13.129+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:13.150 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:13.143+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:13.150 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:13.145+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:13.155 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:13.149+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:13.156 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:13.153+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:13.249 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:13.246+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:13.538 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:13.535+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:13.693 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:13.689+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:14.113 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:14.107+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:14.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:14.118+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:14.137 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:14.131+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:14.167 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:14.161+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:14.170 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:14.165+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:14.258 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:14.254+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:14.544 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:14.540+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:14.671 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:14.667+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:15.070 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:15.065+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:15.096 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:15.091+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:15.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:15.158+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:15.203 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:15.198+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:15.205 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:15.201+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:15.281 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:15.277+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:15.561 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:15.557+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:15.649 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:15.646+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:16.053 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:16.048+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:16.085 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:16.080+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:16.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:16.207+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:16.258 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:16.252+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:16.258 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:16.253+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:16.266 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:16.261+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:16.580 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:16.577+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:16.670 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:16.666+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:17.053 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:17.047+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:17.114 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:17.107+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:17.198 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:17.195+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:17.252 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:17.248+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:17.258 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:17.251+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:17.259 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:17.252+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:17.629 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:17.625+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:17.700 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:17.695+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:18.103 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:18.098+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:18.110 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:18.104+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:18.168 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:18.165+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:18.258 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:18.255+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:18.302 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:18.297+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:18.303 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:18.298+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:18.673 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:18.669+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:18.745 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:18.740+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:19.097 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:19.091+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:19.146 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:19.142+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:19.147 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:19.141+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:19.226 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:19.223+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:19.273 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:19.266+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:19.273 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:19.267+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:19.702 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:19.698+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:19.762 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:19.758+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:20.119 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:20.113+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:20.143 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:20.139+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:20.198 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:20.191+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:20.253 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:20.246+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:20.307 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:20.302+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:20.308 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:20.303+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:20.675 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:20.672+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:20.744 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:20.740+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:21.080 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:21.075+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:21.157 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:21.152+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:21.216 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:21.208+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:21.234 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:21.230+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:21.331 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:21.326+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:21.334 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:21.329+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:21.716 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:21.712+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:21.725 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:21.718+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:21.992 INFO:tasks.ceph.osd.6.smithi181.stderr:*** Caught signal (Segmentation fault) ** 2019-12-08T21:06:21.992 INFO:tasks.ceph.osd.6.smithi181.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:06:22.064 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:22.058+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:22.151 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:22.147+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:22.212 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:22.209+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:22.240 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:22.234+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:22.303 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:22.298+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:22.307 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:22.303+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:22.600 INFO:tasks.ceph.osd.6.smithi181.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:06:22.600 INFO:tasks.ceph.osd.6.smithi181.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:06:22.600 INFO:tasks.ceph.osd.6.smithi181.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:06:22.600 INFO:tasks.ceph.osd.6.smithi181.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:06:22.600 INFO:tasks.ceph.osd.6.smithi181.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:06:22.600 INFO:tasks.ceph.osd.6.smithi181.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:06:22.601 INFO:tasks.ceph.osd.6.smithi181.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:06:22.602 INFO:tasks.ceph.osd.6.smithi181.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:06:22.722 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:22.718+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:22.731 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:22.728+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:23.086 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:23.078+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:23.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:23.143+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:23.241 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:23.237+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:23.289 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:23.284+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:23.329 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:23.324+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:23.332 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:23.327+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:23.417 INFO:tasks.ceph.osd.6.smithi181.stderr:2019-12-08T21:06:23.153+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:06:23.418 INFO:tasks.ceph.osd.6.smithi181.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:06:23.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:06:23.420 INFO:tasks.ceph.osd.6.smithi181.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:06:23.420 INFO:tasks.ceph.osd.6.smithi181.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:06:23.420 INFO:tasks.ceph.osd.6.smithi181.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-08T21:06:23.421 INFO:tasks.ceph.osd.6.smithi181.stderr: 2019-12-08T21:06:23.707 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:23.703+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:23.763 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:23.758+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:24.046 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:24.040+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:24.059 INFO:tasks.ceph.osd.6.smithi181.stderr: -1741> 2019-12-08T21:06:23.153+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-08T21:06:24.059 INFO:tasks.ceph.osd.6.smithi181.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:06:24.059 INFO:tasks.ceph.osd.6.smithi181.stderr: 2019-12-08T21:06:24.059 INFO:tasks.ceph.osd.6.smithi181.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:06:24.059 INFO:tasks.ceph.osd.6.smithi181.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:06:24.059 INFO:tasks.ceph.osd.6.smithi181.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:06:24.060 INFO:tasks.ceph.osd.6.smithi181.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:06:24.060 INFO:tasks.ceph.osd.6.smithi181.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:06:24.060 INFO:tasks.ceph.osd.6.smithi181.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:06:24.060 INFO:tasks.ceph.osd.6.smithi181.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:06:24.060 INFO:tasks.ceph.osd.6.smithi181.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:06:24.060 INFO:tasks.ceph.osd.6.smithi181.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-08T21:06:24.061 INFO:tasks.ceph.osd.6.smithi181.stderr: 2019-12-08T21:06:24.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:24.194+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:24.246 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:24.243+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:24.298 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:24.292+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:24.318 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:24.312+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:24.321 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:24.316+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:24.351 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:06:24.345+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.181:6823/10374,v1:172.21.15.181:6825/10374] for 42sec e379 v379) 2019-12-08T21:06:24.418 INFO:tasks.ceph.osd.6.smithi181.stderr: -1814> 2019-12-08T21:06:23.153+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-08T21:06:24.418 INFO:tasks.ceph.osd.6.smithi181.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:06:24.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 2019-12-08T21:06:24.419 INFO:tasks.ceph.osd.6.smithi181.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:06:24.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:06:24.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:06:24.419 INFO:tasks.ceph.osd.6.smithi181.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:06:24.420 INFO:tasks.ceph.osd.6.smithi181.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:06:24.420 INFO:tasks.ceph.osd.6.smithi181.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:06:24.420 INFO:tasks.ceph.osd.6.smithi181.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:06:24.420 INFO:tasks.ceph.osd.6.smithi181.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:06:24.421 INFO:tasks.ceph.osd.6.smithi181.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:06:24.421 INFO:tasks.ceph.osd.6.smithi181.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:06:24.421 INFO:tasks.ceph.osd.6.smithi181.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:06:24.421 INFO:tasks.ceph.osd.6.smithi181.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:06:24.422 INFO:tasks.ceph.osd.6.smithi181.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:06:24.422 INFO:tasks.ceph.osd.6.smithi181.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:06:24.422 INFO:tasks.ceph.osd.6.smithi181.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-08T21:06:24.422 INFO:tasks.ceph.osd.6.smithi181.stderr: 2019-12-08T21:06:24.731 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:24.728+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:24.732 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:24.728+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:25.080 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:25.074+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:25.211 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:25.208+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:25.266 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:25.262+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:25.337 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:25.331+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:25.380 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:25.333+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:25.380 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:25.334+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:25.753 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:25.750+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:25.766 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:25.762+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:26.133 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:26.127+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:26.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:26.225+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:26.237 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:26.233+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:26.315 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:26.308+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:26.382 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:26.375+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:26.382 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:26.376+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:26.723 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:06:26.719+0000 1694f700 -1 osd.1 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:12.256806+0000 front 2019-12-08T21:05:13.402383+0000 (oldest deadline 2019-12-08T21:05:54.201946+0000) 2019-12-08T21:06:26.792 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:06:26.788+0000 1694f700 -1 osd.2 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.355996+0000 front 2019-12-08T21:05:11.994540+0000 (oldest deadline 2019-12-08T21:05:53.243236+0000) 2019-12-08T21:06:27.108 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:06:27.101+0000 1694f700 -1 osd.4 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.589636+0000 front 2019-12-08T21:05:10.697806+0000 (oldest deadline 2019-12-08T21:05:54.163525+0000) 2019-12-08T21:06:27.245 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:06:27.241+0000 1694f700 -1 osd.0 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.994981+0000 front 2019-12-08T21:05:11.996126+0000 (oldest deadline 2019-12-08T21:05:55.444198+0000) 2019-12-08T21:06:27.273 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:06:27.270+0000 1694f700 -1 osd.3 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.992828+0000 front 2019-12-08T21:05:13.613034+0000 (oldest deadline 2019-12-08T21:05:53.012555+0000) 2019-12-08T21:06:27.317 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:06:27.311+0000 1694f700 -1 osd.5 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:11.991699+0000 front 2019-12-08T21:05:11.961298+0000 (oldest deadline 2019-12-08T21:05:57.029651+0000) 2019-12-08T21:06:27.433 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:27.426+0000 1694f700 -1 osd.7 379 heartbeat_check: no reply from 172.21.15.181:6828 osd.6 since back 2019-12-08T21:05:10.649486+0000 front 2019-12-08T21:05:10.639561+0000 (oldest deadline 2019-12-08T21:05:54.891775+0000) 2019-12-08T21:06:27.433 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:27.428+0000 1694f700 -1 osd.7 379 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:27.561 INFO:tasks.ceph.osd.6.smithi181.stderr:daemon-helper: command crashed with signal 11 2019-12-08T21:06:28.567 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:28.534+0000 1694f700 -1 osd.7 380 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:29.705 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:06:29.701+0000 1694f700 -1 osd.7 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:117980 2.1ds0 2:bb840723:::9161132f-da48-40b4-a853-f341a1971d48.4507.613_%eb%8c%8c%e2%98%83%e8%94%8b%e1%a0%83%e8%94%89%e1%a0%83%e3%9c%86%ed%88%8f%e0%a4%80%e3%9c%86%e3%9c%84%e0%a4%80%e9%90%8c%e1%a0%84%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e379) 2019-12-08T21:06:34.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2019-12-08T21:06:40.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2019-12-08T21:06:46.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~13s 2019-12-08T21:06:49.572 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:23:20.999 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.602 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:23:21.030 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.616 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:23:21.041 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.617 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:23:21.043 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.649 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:23:21.242 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.656 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:23:21.249 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.735 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:23:21.161 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.735 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:23:21.163 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:49.789 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:23:21.382 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:53.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~19s 2019-12-08T21:06:58.224 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:23:29.652 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.231 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:23:29.658 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.260 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:23:29.853 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.263 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:23:29.691 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.267 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:23:29.693 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.271 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:23:29.697 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.281 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:23:29.874 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.309 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:23:29.735 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:58.383 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:23:29.976 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:06:59.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2019-12-08T21:07:05.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~32s 2019-12-08T21:07:12.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~38s 2019-12-08T21:07:18.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~44s 2019-12-08T21:07:24.418 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:24.413+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:24.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~51s 2019-12-08T21:07:25.410 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:25.405+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:25.422 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:25.418+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:25.479 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:25.471+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:26.073 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:26.067+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:26.366 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:26.361+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:26.425 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:26.420+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:26.428 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:26.425+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:26.487 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:26.479+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:26.842 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:26.836+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:27.007 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:27.003+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:27.113 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:27.107+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:27.387 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:27.382+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:27.455 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:27.450+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:27.458 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:27.454+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:27.502 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:27.496+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:27.840 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:27.833+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:28.001 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:27.997+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:28.106 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:28.099+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:28.415 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:28.411+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:28.467 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:28.461+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:28.468 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:28.463+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:28.501 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:28.495+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:28.823 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:28.816+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:29.008 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:29.001+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:29.108 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:29.103+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:29.389 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:29.384+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:29.470 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:29.464+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:29.471 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:29.466+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:29.545 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:29.538+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:29.858 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:29.850+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:29.997 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:29.990+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:30.143 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:30.137+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:30.368 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:30.359+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:30.481 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:30.476+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:30.482 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:30.478+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:30.526 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:30.520+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:30.893 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:30.887+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:30.957 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:30.953+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:31.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~57s 2019-12-08T21:07:31.192 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:31.186+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:31.324 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:31.318+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:31.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:31.433+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:31.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:31.435+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:31.504 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:31.499+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:31.872 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:31.866+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:31.929 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:31.924+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:32.156 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:32.151+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:32.332 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:32.327+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:32.423 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:32.418+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:32.425 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:32.421+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:32.532 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:32.527+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:32.898 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:32.891+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:32.940 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:32.937+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:33.154 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:33.148+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:33.293 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:33.288+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:33.449 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:33.445+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:33.450 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:33.448+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:33.528 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:33.523+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:33.883 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:33.877+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:33.940 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:33.935+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:34.129 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:34.124+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:34.297 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:34.293+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:34.489 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:34.483+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:34.491 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:34.485+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:34.492 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:34.488+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:34.878 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:34.873+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:34.909 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:34.904+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:35.176 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:35.170+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:35.274 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:35.270+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:35.448 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:35.442+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:35.527 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:35.521+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:35.528 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:35.525+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:35.892 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:35.887+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:35.954 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:35.950+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:36.204 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:36.199+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:36.258 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:36.254+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:36.460 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:36.452+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:36.506 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:36.501+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:36.506 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:36.503+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:36.933 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:36.930+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:36.936 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:36.932+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:37.219 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:37.213+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:37.257 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:37.252+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:37.423 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:37.418+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:37.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~63s 2019-12-08T21:07:37.494 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:37.490+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:37.495 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:37.492+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:37.893 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:37.889+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:37.949 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:37.937+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:38.199 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:38.193+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:38.224 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:38.220+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:38.470 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:38.465+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:38.513 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:38.507+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:38.514 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:38.510+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:38.861 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:38.857+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:38.933 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:38.927+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:39.220 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:39.216+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:39.246 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:39.241+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:39.478 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:39.472+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:39.516 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:39.511+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:39.517 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:39.514+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:39.849 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:39.845+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:39.902 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:39.896+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:40.205 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:40.200+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:40.256 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:40.251+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:40.459 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:40.453+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:40.495 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:40.489+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:40.496 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:40.493+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:40.864 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:40.859+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:40.883 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:40.878+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:41.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:41.228+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:41.258 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:41.252+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:41.458 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:41.453+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:41.459 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:41.455+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:41.503 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:41.497+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:41.878 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:41.874+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:41.914 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:41.908+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:42.204 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:42.201+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:42.281 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:42.275+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:42.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:42.446+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:42.452 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:42.449+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:42.465 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:42.453+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:42.876 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:42.872+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:42.882 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:42.876+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:43.251 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:43.245+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:43.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:43.249+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:43.463 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:43.456+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:43.498 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:43.494+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:43.499 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:43.496+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:43.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~69s 2019-12-08T21:07:43.841 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:43.835+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:43.875 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:43.871+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:44.214 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:44.211+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:44.226 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:44.219+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:44.489 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:44.482+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:44.502 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:44.496+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:44.503 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:44.499+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:44.848 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:44.845+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:44.855 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:44.848+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:45.250 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:45.243+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:45.258 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:45.254+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:45.534 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:45.527+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:45.540 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:45.534+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:45.541 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:45.538+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:45.831 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:45.824+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:45.869 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:45.866+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:46.247 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:46.241+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:46.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:46.258+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:46.512 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:46.507+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:46.514 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:46.511+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:46.524 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:46.517+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:46.789 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:46.783+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:46.914 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:46.909+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:47.224 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:47.217+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:47.299 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:47.294+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:47.506 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:47.502+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:47.507 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:47.504+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:47.539 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:47.531+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:47.839 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:47.833+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:47.958 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:47.954+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:48.220 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:48.214+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:48.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:48.250+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:48.513 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:48.507+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:48.537 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:48.532+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:48.539 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:48.535+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:48.878 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:48.868+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:48.921 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:07:48.916+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:41.993778+0000 front 2019-12-08T21:06:42.226261+0000 (oldest deadline 2019-12-08T21:07:26.398454+0000) 2019-12-08T21:07:49.193 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:07:49.187+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.028782+0000 front 2019-12-08T21:06:42.116783+0000 (oldest deadline 2019-12-08T21:07:26.050607+0000) 2019-12-08T21:07:49.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:07:49.229+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:43.180986+0000 front 2019-12-08T21:06:44.150791+0000 (oldest deadline 2019-12-08T21:07:25.197512+0000) 2019-12-08T21:07:49.523 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:07:49.518+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:44.260202+0000 front 2019-12-08T21:06:44.259241+0000 (oldest deadline 2019-12-08T21:07:25.203870+0000) 2019-12-08T21:07:49.534 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:49.530+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.655256+0000 front 2019-12-08T21:06:42.740344+0000 (oldest deadline 2019-12-08T21:07:26.145436+0000) 2019-12-08T21:07:49.534 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:49.532+0000 1694f700 -1 osd.0 381 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:49.902 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:07:49.896+0000 1694f700 -1 osd.4 381 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:06:42.139714+0000 front 2019-12-08T21:06:42.028972+0000 (oldest deadline 2019-12-08T21:07:26.501669+0000) 2019-12-08T21:07:50.090 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~76s 2019-12-08T21:07:50.580 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:50.560+0000 1694f700 -1 osd.0 382 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:51.664 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:07:51.600+0000 1694f700 -1 osd.0 383 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118127 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5418696768 gen 1] snapc 0=[] ondisk+write+known_if_redirected e381) 2019-12-08T21:07:52.086 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:07:52.069+0000 20e25700 -1 RGWWatcher::handle_error cookie 5418696768 err (107) Transport endpoint is not connected 2019-12-08T21:07:56.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~82s 2019-12-08T21:08:02.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~88s 2019-12-08T21:08:09.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2019-12-08T21:08:15.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~101s 2019-12-08T21:08:21.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~107s 2019-12-08T21:08:27.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~114s 2019-12-08T21:08:34.287 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2019-12-08T21:08:40.606 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~126s 2019-12-08T21:08:46.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~133s 2019-12-08T21:08:53.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~139s 2019-12-08T21:08:59.546 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~145s 2019-12-08T21:09:05.874 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~152s 2019-12-08T21:09:07.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:07.220+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:08.241 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:08.236+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:09.250 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:09.247+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:10.225 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:10.215+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:11.205 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:11.201+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:12.173 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:12.170+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:12.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~158s 2019-12-08T21:09:13.132 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:13.128+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:14.157 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:14.155+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:15.182 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:15.180+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:16.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:16.226+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:17.262 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:17.259+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:18.234 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:18.231+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:18.500 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~164s 2019-12-08T21:09:19.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:19.225+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:20.193 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:20.190+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:21.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:21.188+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:22.162 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:22.158+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:23.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:23.206+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:24.193 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:24.190+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:24.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2019-12-08T21:09:25.193 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:25.189+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:26.214 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:26.210+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:27.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:27.253+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:28.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:28.235+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:29.251 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:29.247+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:30.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:30.295+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:31.132 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~177s 2019-12-08T21:09:31.290 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:31.287+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:32.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:32.256+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:33.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:33.254+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:34.226 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:34.223+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:35.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:35.264+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:35.506 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:35.502+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:36.141 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:36.135+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:36.289 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:36.285+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:36.466 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:36.463+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:36.491 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:36.484+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:36.674 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:36.667+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:37.152 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:37.146+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:37.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:37.252+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:37.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~183s 2019-12-08T21:09:37.457 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:37.452+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:37.536 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:37.529+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:37.710 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:37.703+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:38.177 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:38.161+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:38.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:38.224+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:38.470 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:38.466+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:38.544 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:38.539+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:38.684 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:38.678+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:39.209 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:39.202+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:39.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:39.227+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:39.511 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:39.506+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:39.532 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:39.526+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:39.650 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:39.644+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:40.006 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:39.999+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:40.218 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:40.211+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:40.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:40.234+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:40.502 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:40.496+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:40.533 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:40.527+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:40.640 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:40.634+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:40.997 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:40.992+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:41.215 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:41.208+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:41.221 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:41.216+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:41.532 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:41.525+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:41.548 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:41.540+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:41.691 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:41.684+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:41.954 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:41.950+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:42.183 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:42.176+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:42.221 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:42.217+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:42.515 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:42.508+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:42.559 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:42.554+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:42.737 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:42.725+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:42.981 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:42.977+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:43.197 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:43.193+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:43.202 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:43.196+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:43.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:43.548+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:43.581 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:43.575+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:43.710 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:43.704+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:43.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~189s 2019-12-08T21:09:43.988 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:43.981+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:44.187 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:44.179+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:44.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:44.205+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:44.594 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:44.587+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:44.600 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:44.596+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:44.685 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:44.674+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:45.026 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:45.021+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:45.236 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:45.229+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:45.262 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:45.257+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:45.572 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:45.568+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:45.607 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:45.600+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:45.682 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:45.676+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:46.017 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:46.011+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:46.198 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:46.192+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:46.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:46.280+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:46.595 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:46.588+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:46.603 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:46.599+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:46.671 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:46.664+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:47.045 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:47.039+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:47.189 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:47.182+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:47.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:47.297+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:47.565 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:47.559+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:47.578 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:47.573+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:47.693 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:47.686+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:48.025 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:48.022+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:48.177 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:48.170+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:48.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:48.325+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:48.555 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:48.548+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:48.561 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:48.557+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:48.664 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:48.657+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:49.079 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:49.074+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:49.213 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:49.206+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:49.318 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:49.315+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:49.552 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:49.544+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:49.598 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:49.592+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:49.667 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:49.660+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:50.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~196s 2019-12-08T21:09:50.096 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:50.091+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:50.192 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:50.185+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:50.294 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:50.290+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:50.563 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:50.556+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:50.577 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:50.573+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:50.669 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:50.655+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:51.130 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:09:51.125+0000 1694f700 -1 osd.3 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.738906+0000 front 2019-12-08T21:08:53.743679+0000 (oldest deadline 2019-12-08T21:09:39.007789+0000) 2019-12-08T21:09:51.154 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:51.147+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:51.315 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:51.312+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:51.545 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:51.538+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:51.596 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:51.591+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:51.691 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:51.685+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:52.141 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:52.134+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:52.292 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:52.287+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:52.294 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:52.290+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:52.539 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:52.533+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:52.604 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:52.601+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:52.667 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:52.661+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:53.158 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:53.152+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:53.303 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:53.298+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:53.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:53.301+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:53.511 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:53.504+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:53.644 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:53.640+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:53.687 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:53.681+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:54.160 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:54.153+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:54.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:54.346+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:54.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:54.348+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:54.528 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:54.522+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:54.654 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:54.651+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:54.705 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:54.699+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:55.162 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:55.155+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:55.361 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:55.358+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:55.362 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:55.359+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:55.502 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:55.496+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:55.657 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:55.654+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:55.703 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:55.697+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:56.137 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:56.131+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:56.386 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~202s 2019-12-08T21:09:56.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:56.395+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:56.402 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:56.398+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:56.543 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:56.537+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:56.699 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:56.696+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:56.701 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:56.694+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:57.114 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:57.108+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:57.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:57.406+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:57.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:57.408+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:57.519 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:57.514+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:57.681 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:57.677+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:57.711 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:57.705+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:58.076 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:58.070+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:58.461 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:58.457+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:58.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:58.459+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:58.479 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:58.473+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:58.690 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:58.687+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:58.761 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:58.754+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:09:59.043 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:09:59.037+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:09:59.421 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:59.417+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:09:59.423 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:09:59.419+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:09:59.517 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:09:59.511+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:09:59.717 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:09:59.714+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:09:59.720 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:09:59.715+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:10:00.012 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:10:00.005+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:10:00.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:00.458+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:10:00.463 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:00.460+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:00.489 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:10:00.480+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:10:00.750 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:10:00.744+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:10:00.768 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:10:00.764+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:10:01.021 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:10:01.015+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:10:01.497 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:10:01.485+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:10:01.505 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:01.499+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:10:01.506 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:01.502+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:01.750 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:10:01.743+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:10:01.805 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:10:01.788+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:10:02.046 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:10:02.039+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:10:02.485 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:02.479+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:10:02.486 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:02.482+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:02.508 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:10:02.501+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:10:02.711 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~208s 2019-12-08T21:10:02.730 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:10:02.723+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:10:02.784 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:10:02.779+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:10:03.001 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:10:02.995+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:10:03.518 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:03.512+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:10:03.519 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:03.516+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:03.541 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:10:03.535+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:10:03.758 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:10:03.750+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:10:03.830 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:10:03.826+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:10:03.966 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:10:03.960+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:10:04.483 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:04.478+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.491553+0000 front 2019-12-08T21:08:53.490888+0000 (oldest deadline 2019-12-08T21:09:52.178898+0000) 2019-12-08T21:10:04.483 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:04.480+0000 1694f700 -1 osd.0 387 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:04.537 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:10:04.530+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.246768+0000 front 2019-12-08T21:08:52.248573+0000 (oldest deadline 2019-12-08T21:09:35.880608+0000) 2019-12-08T21:10:04.730 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:10:04.724+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:52.094530+0000 front 2019-12-08T21:08:51.706439+0000 (oldest deadline 2019-12-08T21:09:35.758770+0000) 2019-12-08T21:10:04.798 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:10:04.794+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:49.929781+0000 front 2019-12-08T21:08:49.549862+0000 (oldest deadline 2019-12-08T21:09:34.756404+0000) 2019-12-08T21:10:04.964 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:10:04.957+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:08:53.512684+0000 front 2019-12-08T21:08:53.513917+0000 (oldest deadline 2019-12-08T21:09:35.119701+0000) 2019-12-08T21:10:05.626 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:05.475+0000 1694f700 -1 osd.0 388 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:06.439 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:06.435+0000 1694f700 -1 osd.0 388 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4530.0:118299 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096] snapc 0=[] ondisk+write+known_if_redirected e384) 2019-12-08T21:10:07.789 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:10:07.627+0000 1694f700 -1 osd.0 389 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118306 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 5422538096 gen 1] snapc 0=[] ondisk+write+known_if_redirected e386) 2019-12-08T21:10:09.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2019-12-08T21:10:15.358 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~221s 2019-12-08T21:10:21.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~227s 2019-12-08T21:10:27.984 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~234s 2019-12-08T21:10:34.297 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2019-12-08T21:10:40.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~246s 2019-12-08T21:10:46.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~253s 2019-12-08T21:10:53.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~259s 2019-12-08T21:10:58.462 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:27:29.890 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.468 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:27:29.896 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.485 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:27:29.911 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.488 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:27:29.915 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.506 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:27:30.099 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.529 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:27:30.122 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.764 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:27:30.190 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.777 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:27:30.195 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:58.817 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:27:30.410 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:10:59.541 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2019-12-08T21:11:05.852 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~272s 2019-12-08T21:11:12.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~278s 2019-12-08T21:11:18.477 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~284s 2019-12-08T21:11:24.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~290s 2019-12-08T21:11:31.102 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~297s 2019-12-08T21:11:37.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~303s 2019-12-08T21:11:37.418 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-12-08T21:11:37.418 ERROR:tasks.daemonwatchdog.daemon_watchdog:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 38, in _run self.watch() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 114, in watch self.bark() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 54, in bark for mount in self.ctx.mounts.values(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/config.py", line 253, in __getattr__ raise AttributeError(name) AttributeError: mounts 2019-12-08T21:11:44.413 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:44.407+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:45.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:45.325+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:45.447 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:45.444+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:45.529 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:45.525+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:46.279 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:46.273+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:46.308 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:46.304+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:46.418 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:46.414+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:46.570 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:46.566+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:47.209 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:47.202+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:47.260 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:47.253+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:47.295 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:47.288+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:47.442 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:47.437+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:47.591 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:47.586+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:47.751 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:47.744+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:48.201 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:48.194+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:48.271 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:48.265+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:48.321 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:48.317+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:48.485 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:48.480+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:48.566 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:48.561+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:48.728 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:48.722+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:49.201 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:49.193+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:49.296 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:49.290+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:49.307 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:49.303+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:49.451 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:49.448+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:49.597 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:49.593+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:49.692 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:49.688+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:50.192 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:50.185+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:50.289 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:50.284+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:50.292 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:50.289+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:50.318 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:50.312+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:50.478 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:50.474+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:50.604 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:50.601+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:50.651 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:50.646+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:51.205 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:51.199+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:51.272 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:51.267+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:51.273 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:51.270+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:51.301 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:51.296+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:51.502 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:51.498+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:51.617 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:51.613+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:51.621 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:51.615+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:52.225 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:52.218+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:52.264 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:52.260+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:52.265 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:52.262+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:52.330 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:52.326+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:52.554 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:52.548+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:52.633 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:52.628+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:52.635 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:52.632+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:53.198 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:53.192+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:53.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:53.254+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:53.258 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:53.255+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:53.338 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:53.332+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:53.606 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:53.598+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:53.606 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:53.601+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:53.611 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:53.607+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:54.227 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:54.222+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:54.289 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:54.283+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:54.289 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:54.285+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:54.350 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:54.345+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:54.563 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:54.560+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:54.629 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:54.625+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:54.639 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:54.634+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:55.244 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:55.238+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:55.307 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:55.302+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:55.308 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:55.304+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:55.403 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:55.397+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:55.596 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:55.591+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:55.615 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:55.611+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:55.685 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:55.679+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:56.206 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:56.200+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:56.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:56.301+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:56.307 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:56.303+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:56.397 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:56.392+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:56.607 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:56.603+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:56.664 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:56.660+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:56.729 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:56.723+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:57.259 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:57.253+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:57.358 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:57.352+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:57.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:57.355+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:57.422 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:57.417+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:57.642 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:57.637+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:57.698 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:57.693+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:57.725 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:57.718+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:58.274 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:58.269+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:58.400 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:58.393+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:58.403 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:58.398+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:58.446 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:58.441+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:58.621 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:58.615+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:58.681 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:58.675+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:58.732 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:58.726+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:11:59.289 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:11:59.284+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:11:59.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:59.434+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:11:59.440 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:11:59.436+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:11:59.451 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:11:59.445+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:11:59.646 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:11:59.641+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:11:59.696 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:11:59.691+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:11:59.706 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:11:59.700+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:00.322 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:00.316+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:00.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:00.407+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:00.414 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:00.409+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:00.465 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:00.459+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:00.615 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:00.611+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:00.666 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:00.661+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:00.738 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:00.730+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:01.322 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:01.316+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:01.418 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:01.410+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:01.419 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:01.414+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:01.473 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:01.468+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:01.600 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:01.595+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:01.718 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:01.713+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:01.752 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:01.747+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:02.341 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:02.335+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:02.376 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:02.371+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:02.378 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:02.374+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:02.507 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:02.501+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:02.603 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:02.589+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:02.725 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:02.718+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:02.741 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:02.736+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:03.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:03.368+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:03.375 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:03.371+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:03.393 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:03.387+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:03.516 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:03.510+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:03.631 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:03.625+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:03.754 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:03.747+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:03.756 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:03.751+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:04.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:04.406+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:04.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:04.408+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:04.439 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:04.433+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:04.536 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:04.530+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:04.651 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:04.647+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:04.763 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:04.760+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:04.802 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:04.796+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:05.387 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:05.382+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:05.397 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:05.394+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:05.475 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:05.463+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:05.591 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:05.586+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:05.605 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:05.603+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:05.721 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:05.717+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:05.818 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:05.807+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:06.381 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:06.376+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:06.382 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:06.378+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:06.444 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:06.438+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:06.559 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:06.552+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:06.589 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:06.585+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:06.771 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:06.767+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:06.781 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:06.775+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:07.433 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:07.428+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:07.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:07.430+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:07.473 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:07.459+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:07.585 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:07.578+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:07.593 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:07.589+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:07.779 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:07.775+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:07.795 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:07.789+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:08.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:08.404+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:08.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:08.406+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:08.431 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:08.424+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:08.566 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:08.559+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:08.620 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:08.616+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:08.774 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:08.767+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:08.830 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:08.826+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:09.425 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:09.419+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:09.436 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:09.431+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:09.437 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:09.434+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:09.616 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:09.610+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:09.622 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:09.618+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:09.786 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:09.779+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:09.868 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:09.864+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:10.388 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:10.382+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:10.408 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:10.404+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:10.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:10.406+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:10.574 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:10.568+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:10.620 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:10.616+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:10.792 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:10.786+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:10.832 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:10.827+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:11.376 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:11.367+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:11.377 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:11.369+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:11.439 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:11.432+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:11.579 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:11.576+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:11.586 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:11.579+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:11.803 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:11.799+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:11.808 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:11.802+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:12.414 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:12.410+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:12.416 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:12.412+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:12.417 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:12.411+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:12.564 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:12.561+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:12.579 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:12.572+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:12.788 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:12.783+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:12.831 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:12.823+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:13.409 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:13.402+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:13.419 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:13.414+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:13.420 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:13.417+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:13.550 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:13.542+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:13.579 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:13.573+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:13.775 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:13.771+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:13.786 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:13.780+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:14.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:14.408+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:14.414 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:14.410+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:14.418 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:14.412+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:14.569 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:14.565+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:14.578 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:14.573+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:14.803 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:14.797+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:14.818 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:14.813+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:15.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:12:15.378+0000 1694f700 -1 osd.5 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.128001+0000 front 2019-12-08T21:11:01.121141+0000 (oldest deadline 2019-12-08T21:11:46.934739+0000) 2019-12-08T21:12:15.466 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:15.459+0000 1694f700 -1 osd.0 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.631217+0000 front 2019-12-08T21:11:01.836071+0000 (oldest deadline 2019-12-08T21:11:45.091961+0000) 2019-12-08T21:12:15.466 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:15.463+0000 1694f700 -1 osd.0 393 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:15.542 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:12:15.539+0000 1694f700 -1 osd.2 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:10:59.932758+0000 front 2019-12-08T21:10:59.912858+0000 (oldest deadline 2019-12-08T21:11:43.968412+0000) 2019-12-08T21:12:15.626 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:12:15.620+0000 1694f700 -1 osd.4 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:00.299396+0000 front 2019-12-08T21:11:00.318992+0000 (oldest deadline 2019-12-08T21:11:46.184842+0000) 2019-12-08T21:12:15.845 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:12:15.841+0000 1694f700 -1 osd.3 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:03.071680+0000 front 2019-12-08T21:11:03.587625+0000 (oldest deadline 2019-12-08T21:11:44.741442+0000) 2019-12-08T21:12:15.857 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:12:15.851+0000 1694f700 -1 osd.7 393 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:11:01.529221+0000 front 2019-12-08T21:11:01.536330+0000 (oldest deadline 2019-12-08T21:11:47.396311+0000) 2019-12-08T21:12:16.596 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:16.535+0000 1694f700 -1 osd.0 394 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:17.659 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:12:17.560+0000 1694f700 -1 osd.0 395 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118670 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5422538096 gen 3] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-08T21:12:18.289 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:12:18.286+0000 20e25700 -1 RGWWatcher::handle_error cookie 5422538096 err (107) Transport endpoint is not connected 2019-12-08T21:13:23.123 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:23.120+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:24.076 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:24.073+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:25.107 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:25.104+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:26.095 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:26.092+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:27.086 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:27.084+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:28.119 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:28.114+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:29.122 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:29.118+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:30.091 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:30.086+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:31.085 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:31.080+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:32.084 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:32.080+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:33.050 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:33.047+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:34.067 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:34.063+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:35.063 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:35.059+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:36.093 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:36.090+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:37.124 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:37.122+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:38.127 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:38.125+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:39.103 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:39.100+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:40.080 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:40.077+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:41.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:41.101+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:42.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:42.137+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:43.148 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:43.145+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:44.140 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:44.137+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:45.183 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:45.179+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:46.136 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:46.133+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:47.108 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:47.105+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:48.094 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:48.091+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:49.139 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:49.136+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:50.183 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:50.180+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:50.829 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:50.826+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:51.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:51.218+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:51.225 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:51.222+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:51.716 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:51.710+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:51.787 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:51.783+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:52.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:52.251+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:52.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:52.253+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:52.300 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:52.293+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:52.338 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:52.329+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:52.481 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:52.477+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:52.708 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:52.701+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:52.815 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:52.811+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:53.255 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:53.248+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:53.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:53.255+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:53.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:53.257+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:53.370 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:53.363+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:53.493 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:53.488+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:53.753 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:53.747+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:53.838 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:53.833+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:54.253 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:54.249+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:54.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:54.253+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:54.305 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:54.299+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:54.331 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:54.325+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:54.481 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:54.477+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:54.790 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:54.783+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:54.858 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:54.853+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:55.262 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:55.256+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:55.280 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:55.276+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:55.280 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:55.278+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:55.336 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:55.329+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:55.511 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:55.505+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:55.819 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:55.812+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:55.897 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:55.894+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:56.268 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:56.262+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:56.314 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:56.310+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:56.315 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:56.312+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:56.347 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:56.341+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:56.523 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:56.518+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:56.846 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:56.839+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:56.850 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:56.847+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:57.270 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:57.265+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:57.270 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:57.267+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:57.276 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:57.269+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:57.327 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:57.320+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:57.556 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:57.552+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:57.862 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:57.856+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:57.899 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:57.896+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:58.286 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:58.281+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:58.296 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:58.285+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:58.296 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:58.287+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:58.356 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:58.349+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:58.586 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:58.583+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:58.869 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:58.863+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:58.942 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:58.938+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:13:59.297 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:59.292+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:13:59.303 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:13:59.297+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:13:59.307 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:13:59.301+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:13:59.404 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:13:59.398+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:13:59.547 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:13:59.543+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:13:59.864 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:13:59.858+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:13:59.929 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:13:59.924+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:00.318 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:00.308+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:00.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:00.330+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:00.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:00.331+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:00.404 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:00.398+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:00.587 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:00.583+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:00.900 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:00.894+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:00.966 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:00.962+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:01.297 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:01.290+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:01.303 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:01.299+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:01.304 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:01.301+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:01.453 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:01.446+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:01.547 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:01.543+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:01.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:01.870+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:02.016 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:02.012+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:02.311 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:02.304+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:02.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:02.345+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:02.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:02.347+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:02.421 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:02.415+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:02.507 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:02.504+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:02.890 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:02.883+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:02.976 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:02.970+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:03.290 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:03.280+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:03.361 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:03.356+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:03.364 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:03.362+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:03.437 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:03.430+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:03.513 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:03.508+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:03.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:03.850+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:03.982 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:03.979+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:04.251 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:04.243+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:04.320 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:04.316+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:04.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:04.320+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:04.446 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:04.440+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:04.556 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:04.550+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:04.841 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:04.834+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:04.996 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:04.992+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:05.237 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:05.228+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:05.369 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:05.365+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:05.373 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:05.370+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:05.500 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:05.493+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:05.517 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:05.512+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:05.885 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:05.879+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:06.033 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:06.030+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:06.288 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:06.279+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:06.343 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:06.339+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:06.344 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:06.341+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:06.492 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:06.485+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:06.563 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:06.557+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:06.859 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:06.846+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:07.065 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:07.061+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:07.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:07.307+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:07.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:07.330+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:07.335 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:07.332+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:07.519 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:07.513+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:07.611 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:07.607+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:07.839 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:07.833+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:08.053 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:08.050+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:08.328 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:08.322+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:08.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:08.325+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:08.364 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:08.357+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:08.487 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:08.480+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:08.590 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:08.586+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:08.796 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:08.789+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:09.026 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:09.023+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:09.284 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:09.278+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:09.284 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:09.281+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:09.368 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:09.362+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:09.486 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:09.478+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:09.601 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:09.597+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:09.786 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:09.780+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:10.062 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:10.058+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:10.320 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:10.315+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:10.321 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:10.317+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:10.389 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:10.382+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:10.533 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:10.524+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:10.588 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:10.584+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:10.824 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:10.817+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:11.015 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:11.011+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:11.327 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:11.323+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:11.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:11.325+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:11.379 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:11.371+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:11.562 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:11.555+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:11.588 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:11.584+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:11.825 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:11.819+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:11.970 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:11.965+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:12.321 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:12.315+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:12.323 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:12.319+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:12.338 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:12.332+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:12.541 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:12.528+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:12.556 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:12.552+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:12.835 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:12.828+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:13.019 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:13.016+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:13.339 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:13.333+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:13.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:13.338+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:13.392 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:13.385+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:13.512 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:13.506+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:13.538 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:13.529+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:13.850 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:13.844+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:14.052 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:14.047+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:14.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:14.306+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:14.314 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:14.310+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:14.406 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:14.395+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:14.504 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:14.498+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:14.545 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:14.540+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:14.853 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:14.846+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:15.017 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:15.012+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:15.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:15.353+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:15.363 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:15.359+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:15.398 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:15.391+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:15.561 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:15.554+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:15.583 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:15.578+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:15.808 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:15.802+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:16.022 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:16.017+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:16.386 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:16.381+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:16.389 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:16.385+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:16.445 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:16.439+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:16.607 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:16.601+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:16.609 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:16.603+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:16.799 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:16.788+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:17.033 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:17.029+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:17.425 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:17.418+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:17.426 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:17.420+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:17.470 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:17.464+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:17.578 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:17.574+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:17.587 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:17.581+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:17.777 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:17.754+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:18.002 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:17.997+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:18.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:18.452+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:18.458 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:18.454+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:18.481 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:18.472+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:18.569 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:18.561+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:18.624 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:18.618+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:18.717 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:18.711+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:19.036 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:19.031+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:19.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:19.452+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:19.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:19.457+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:19.529 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:19.522+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:19.540 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:19.534+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:19.642 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:19.637+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:19.679 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:19.672+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:20.009 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:20.004+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:20.497 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:20.492+0000 1694f700 -1 osd.0 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:04.552123+0000 front 2019-12-08T21:13:04.464898+0000 (oldest deadline 2019-12-08T21:13:50.355100+0000) 2019-12-08T21:14:20.500 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:20.495+0000 1694f700 -1 osd.0 399 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:20.513 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:20.506+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:20.560 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:14:20.554+0000 1694f700 -1 osd.7 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.941956+0000 front 2019-12-08T21:13:07.938259+0000 (oldest deadline 2019-12-08T21:13:52.013141+0000) 2019-12-08T21:14:20.646 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:14:20.634+0000 1694f700 -1 osd.3 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.684520+0000 front 2019-12-08T21:13:07.696712+0000 (oldest deadline 2019-12-08T21:13:52.325380+0000) 2019-12-08T21:14:20.693 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:14:20.684+0000 1694f700 -1 osd.5 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:08.581495+0000 front 2019-12-08T21:13:08.527635+0000 (oldest deadline 2019-12-08T21:13:51.346617+0000) 2019-12-08T21:14:21.060 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:14:21.055+0000 1694f700 -1 osd.2 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:06.144925+0000 front 2019-12-08T21:13:06.096904+0000 (oldest deadline 2019-12-08T21:13:50.681241+0000) 2019-12-08T21:14:21.234 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:14:21.227+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 46sec e399 v399) 2019-12-08T21:14:21.541 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:14:21.537+0000 1694f700 -1 osd.4 399 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:13:07.714695+0000 front 2019-12-08T21:13:07.719833+0000 (oldest deadline 2019-12-08T21:13:51.750237+0000) 2019-12-08T21:14:22.106 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:22.100+0000 1694f700 -1 osd.0 400 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:23.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:14:23.171+0000 1694f700 -1 osd.0 401 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4530.0:118838 5.5 5.88aa5c95 (undecoded) ondisk+write+known_if_redirected e397) 2019-12-08T21:14:56.537 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:31:27.965 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.547 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:31:27.975 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.556 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:31:27.982 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.559 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:31:27.985 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.565 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:31:28.158 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.587 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:31:28.180 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.827 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:31:28.253 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.881 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:31:28.310 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:14:56.947 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:31:28.540 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:04.311 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:15:04.292+0000 34f8d700 -1 osd.1 404 build_incremental_map_msg missing incremental map 404 2019-12-08T21:15:04.334 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:15:04.324+0000 124b0700 -1 osd.1 404 build_incremental_map_msg missing incremental map 404 2019-12-08T21:15:04.621 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:15:04.560+0000 11caf700 -1 osd.1 404 build_incremental_map_msg missing incremental map 404 2019-12-08T21:15:04.925 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:15:04.856+0000 34f8d700 -1 osd.1 404 build_incremental_map_msg missing incremental map 404 2019-12-08T21:15:04.957 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:15:04.888+0000 33f8b700 -1 osd.1 404 build_incremental_map_msg missing incremental map 404 2019-12-08T21:15:08.369 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:15:08.357+0000 3378a700 -1 osd.1 405 build_incremental_map_msg missing incremental map 404 2019-12-08T21:15:11.496 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:31:42.922 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.498 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:31:42.926 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.519 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:31:42.947 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.536 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:31:42.962 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.579 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:31:43.005 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.593 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:31:43.186 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.595 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:31:43.023 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.602 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:31:43.196 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:11.683 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:31:43.276 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:15:33.318 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:33.315+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:34.361 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:34.357+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:35.392 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:35.389+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:36.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:36.410+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:37.394 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:37.391+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:38.411 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:38.408+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:39.412 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:39.409+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:40.381 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:40.376+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:41.373 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:41.369+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:42.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:42.355+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:43.323 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:43.320+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:44.369 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:44.365+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:45.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:45.325+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:46.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:46.369+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:47.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:47.330+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:48.297 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:48.293+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:49.261 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:49.253+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:50.237 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:50.233+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:51.261 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:51.253+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:52.215 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:52.210+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:53.192 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:53.189+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:54.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:54.226+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:55.185 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:55.182+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:56.184 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:56.181+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:57.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:57.189+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:58.168 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:58.164+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:15:59.159 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:15:59.156+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:00.181 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:00.179+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:01.188 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:01.186+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:02.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:02.225+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:03.201 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:03.198+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:04.224 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:04.221+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:05.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:05.186+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:05.411 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:05.407+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:05.439 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:05.432+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:05.501 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:05.494+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:06.232 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:06.228+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:06.233 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:06.229+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:06.337 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:06.328+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:06.413 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:06.404+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:06.458 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:06.454+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:06.524 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:06.516+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:07.268 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:07.263+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:07.280 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:07.271+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:07.344 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:07.339+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:07.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:07.379+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:07.485 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:07.478+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:07.509 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:07.503+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:08.233 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:08.229+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:08.234 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:08.231+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:08.272 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:08.266+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:08.343 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:08.338+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:08.364 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:08.358+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:08.446 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:08.442+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:08.484 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:08.478+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:09.255 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:09.250+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:09.256 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:09.253+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:09.319 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:09.315+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:09.361 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:09.356+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:09.413 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:09.407+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:09.418 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:09.413+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:09.486 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:09.482+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:10.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:10.234+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:10.239 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:10.236+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:10.315 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:10.312+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:10.345 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:10.341+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:10.445 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:10.439+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:10.463 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:10.458+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:10.472 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:10.464+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:11.238 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:11.235+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:11.239 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:11.236+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:11.318 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:11.311+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:11.367 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:11.363+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:11.411 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:11.406+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:11.473 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:11.468+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:11.485 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:11.481+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:12.271 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:12.266+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:12.272 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:12.269+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:12.366 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:12.361+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:12.377 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:12.373+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:12.430 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:12.422+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:12.475 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:12.471+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:12.487 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:12.482+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:13.317 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:13.311+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:13.317 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:13.314+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:13.372 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:13.367+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:13.380 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:13.374+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:13.432 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:13.426+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:13.496 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:13.490+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:13.511 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:13.507+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:14.287 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:14.280+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:14.288 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:14.283+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:14.423 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:14.418+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:14.429 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:14.424+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:14.451 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:16:14.446+0000 1694f700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.817287+0000 front 2019-12-08T21:15:23.489681+0000 (oldest deadline 2019-12-08T21:16:05.689711+0000) 2019-12-08T21:16:14.455 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:14.450+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:14.548 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:14.543+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:15.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:15.262+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:15.267 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:15.264+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:15.389 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:15.385+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:15.438 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:15.433+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:15.447 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:15.442+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:15.593 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:15.590+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:16.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:16.253+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:16.259 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:16.256+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:16.417 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:16.414+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:16.426 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:16.420+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:16.464 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:16.458+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:16.641 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:16.637+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:17.290 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:17.285+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:17.291 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:17.288+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:17.444 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:17.437+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:17.446 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:17.444+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:17.464 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:17.456+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:17.643 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:17.639+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:18.316 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:18.308+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:18.318 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:18.311+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:18.413 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:18.409+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:18.433 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:18.428+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:18.482 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:18.476+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:18.643 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:18.640+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:19.332 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:19.320+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:19.332 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:19.323+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:19.373 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:19.368+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:19.443 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:19.435+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:19.447 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:19.442+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:19.671 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:19.668+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:20.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:20.280+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:20.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:20.282+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:20.384 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:20.378+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:20.416 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:20.410+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:20.433 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:20.427+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:20.658 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:20.655+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:21.312 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:21.307+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:21.313 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:21.310+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:21.384 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:21.380+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:21.388 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:21.382+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:21.432 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:21.425+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:21.711 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:21.707+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:22.358 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:22.353+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:22.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:22.356+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:22.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:22.393+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:22.414 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:22.407+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:22.435 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:22.428+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:22.736 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:22.731+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:23.407 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:23.400+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:23.407 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:23.401+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:23.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:23.404+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:23.410 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:23.407+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:23.433 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:23.426+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:23.744 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:23.740+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:24.379 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:24.371+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:24.390 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:24.384+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:24.438 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:24.433+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:24.439 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:24.435+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:24.444 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:24.442+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:24.759 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:24.756+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:25.333 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:25.327+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:25.358 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:25.351+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:25.453 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:25.449+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:25.454 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:25.451+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:25.482 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:25.476+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:25.772 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:25.768+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:26.301 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:26.296+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:26.388 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:26.382+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:26.496 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:26.492+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:26.499 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:26.494+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:26.501 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:26.497+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:26.817 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:26.814+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:27.353 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:27.347+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:27.369 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:27.359+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:27.452 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:27.448+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:27.520 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:27.514+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:27.521 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:27.516+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:27.849 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:27.844+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:28.319 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:28.312+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:28.321 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:28.315+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:28.486 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:28.482+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:28.565 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:28.560+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:28.567 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:28.563+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:28.897 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:28.893+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:29.329 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:29.322+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:29.339 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:29.333+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:29.522 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:29.517+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:29.594 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:29.589+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:29.595 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:29.591+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:29.906 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:29.900+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:30.375 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:30.368+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:30.389 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:30.382+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:30.529 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:30.524+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:30.624 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:30.618+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:30.632 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:30.620+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:30.936 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:30.931+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:31.329 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:31.322+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:31.363 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:31.357+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:31.493 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:31.488+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:31.637 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:31.628+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:31.637 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:31.631+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:31.980 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:31.976+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:32.326 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:32.319+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:32.371 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:32.365+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:32.476 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:32.471+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:32.600 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:32.594+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:32.601 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:32.597+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:33.011 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:33.007+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:33.360 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:33.353+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:33.379 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:33.373+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:33.490 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:33.486+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:33.556 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:33.550+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:33.557 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:33.553+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:34.008 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:34.002+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:34.383 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:34.376+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:34.401 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:34.395+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:34.444 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:34.439+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:34.604 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:34.599+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:34.604 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:34.601+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:35.031 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:35.026+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:35.365 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:35.358+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:35.433 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:35.424+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:35.440 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:35.434+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:35.566 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:35.558+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:35.567 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:35.561+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:36.082 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:36.077+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:36.400 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:36.394+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:36.415 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:36.408+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:36.459 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:36.453+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:36.567 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:36.561+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:36.569 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:36.564+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:36.754 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:16:36.750+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e405 v405) 2019-12-08T21:16:37.076 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:37.072+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:37.386 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:37.379+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:37.431 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:37.425+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:37.450 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:37.445+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:37.597 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:37.592+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:37.598 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:37.594+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:38.065 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:38.057+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:38.410 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:38.403+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:38.421 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:38.417+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:38.477 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:38.470+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:38.557 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:38.551+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:38.558 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:38.554+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:39.063 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:39.059+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:39.380 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:39.374+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:39.450 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:39.446+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:39.523 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:39.516+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:39.608 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:39.602+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:39.609 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:39.606+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:40.030 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:40.024+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:40.392 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:40.386+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:40.493 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:40.486+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:40.570 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:40.565+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:40.644 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:40.640+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:40.645 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:40.642+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:41.003 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:16:40.999+0000 1694f700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:21.010282+0000 front 2019-12-08T21:15:21.069749+0000 (oldest deadline 2019-12-08T21:16:05.063484+0000) 2019-12-08T21:16:41.421 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:16:41.415+0000 1694f700 -1 osd.4 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.665078+0000 front 2019-12-08T21:15:23.531558+0000 (oldest deadline 2019-12-08T21:16:05.161326+0000) 2019-12-08T21:16:41.514 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:16:41.509+0000 1694f700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:22.161610+0000 front 2019-12-08T21:15:22.165394+0000 (oldest deadline 2019-12-08T21:16:06.182926+0000) 2019-12-08T21:16:41.565 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:16:41.558+0000 1694f700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:23.396125+0000 front 2019-12-08T21:15:23.532276+0000 (oldest deadline 2019-12-08T21:16:05.002534+0000) 2019-12-08T21:16:41.616 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:41.611+0000 1694f700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:15:24.027460+0000 front 2019-12-08T21:15:24.056363+0000 (oldest deadline 2019-12-08T21:16:08.103467+0000) 2019-12-08T21:16:41.618 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:41.614+0000 1694f700 -1 osd.0 405 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:41.816 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:16:41.811+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e405 v405) 2019-12-08T21:16:42.686 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:42.682+0000 1694f700 -1 osd.0 406 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:43.726 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:16:43.647+0000 1694f700 -1 osd.0 407 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119160 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 904141568 gen 3] snapc 0=[] ondisk+write+known_if_redirected e403) 2019-12-08T21:16:44.250 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:16:44.227+0000 20e25700 -1 RGWWatcher::handle_error cookie 904141568 err (107) Transport endpoint is not connected 2019-12-08T21:17:07.327 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:33:38.754 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.353 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:33:38.782 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.360 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:33:38.954 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.362 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:33:38.955 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.363 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:33:38.787 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.413 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:33:38.839 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.537 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:33:39.130 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.676 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:33:39.104 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:07.701 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:33:39.127 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:17:15.409 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:17:15.402+0000 11caf700 -1 osd.1 422 build_incremental_map_msg missing incremental map 422 2019-12-08T21:17:15.947 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:17:15.884+0000 30f85700 -1 osd.1 422 build_incremental_map_msg missing incremental map 422 2019-12-08T21:17:15.969 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:17:15.913+0000 31786700 -1 osd.1 422 build_incremental_map_msg missing incremental map 422 2019-12-08T21:17:16.342 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:17:16.312+0000 3378a700 -1 osd.1 422 build_incremental_map_msg missing incremental map 422 2019-12-08T21:17:24.405 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:17:24.401+0000 32f89700 -1 osd.1 429 build_incremental_map_msg missing incremental map 404 2019-12-08T21:17:24.435 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:17:24.431+0000 32f89700 -1 osd.1 429 build_incremental_map_msg missing incremental map 422 2019-12-08T21:18:08.346 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:08.341+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:08.848 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:34:40.276 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:08.857 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:34:40.285 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:08.870 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:34:40.296 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:08.874 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:34:40.300 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:08.878 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:34:40.472 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:08.894 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:34:40.487 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:08.972 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:34:40.400 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:09.003 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:34:40.429 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:09.141 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:34:40.734 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:18:09.306 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:09.302+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:10.306 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:10.301+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:11.270 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:11.265+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:12.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:12.226+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:13.213 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:13.209+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:14.244 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:14.239+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:15.202 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:15.197+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:15.348 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:15.345+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:16.223 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:16.218+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:16.318 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:16.314+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:17.223 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:17.218+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:17.345 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:17.341+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:18.220 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:18.215+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:18.349 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:18.346+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:19.261 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:19.256+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:19.338 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:19.336+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:20.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:20.226+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:20.375 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:20.372+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:20.900 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:20.893+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:21.235 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:21.230+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:21.333 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:21.329+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:21.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:21.331+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:21.871 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:21.869+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:22.278 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:22.273+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:22.380 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:22.376+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:22.380 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:22.377+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:22.704 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:22.698+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:22.910 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:22.906+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:23.137 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:23.130+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:23.262 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:23.259+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:23.265 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:23.260+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:23.352 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:23.347+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:23.352 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:23.349+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:23.713 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:23.707+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:23.919 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:23.914+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:24.163 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:24.157+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:24.224 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:24.219+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:24.238 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:24.234+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:24.335 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:24.331+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:24.336 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:24.332+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:24.757 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:24.750+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:24.901 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:24.898+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:25.133 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:25.124+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:25.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:25.224+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:25.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:25.227+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:25.250 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:25.247+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:25.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:25.368+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:25.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:25.371+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:25.756 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:25.750+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:25.907 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:25.904+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:26.171 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:26.165+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:26.254 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:26.249+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:26.271 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:26.264+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:26.272 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:26.267+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:26.350 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:26.345+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:26.351 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:26.348+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:26.809 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:26.803+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:26.877 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:26.874+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:27.151 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:27.144+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:27.218 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:27.215+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:27.252 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:27.246+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:27.252 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:27.248+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:27.399 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:27.393+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:27.399 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:27.396+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:27.805 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:27.799+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:27.835 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:27.831+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:28.165 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:28.158+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:28.191 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:28.186+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:28.298 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:28.291+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:28.299 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:28.294+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:28.364 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:28.359+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:28.365 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:28.362+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:28.826 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:28.820+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:28.844 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:28.840+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:29.144 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:29.138+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:29.148 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:29.145+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:29.293 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:29.286+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:29.294 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:29.289+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:29.346 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:29.342+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:29.347 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:29.344+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:29.808 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:29.805+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:29.821 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:29.815+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:30.102 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:30.099+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:30.104 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:30.097+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:30.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:30.320+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:30.325 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:30.322+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:30.337 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:30.326+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:30.337 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:30.327+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:30.792 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:30.785+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:30.824 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:30.821+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:31.072 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:31.068+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:31.086 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:31.079+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:31.297 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:31.290+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:31.300 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:31.294+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:31.331 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:31.326+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:31.332 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:31.328+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:31.804 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:31.797+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:31.830 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:31.826+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:32.096 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:32.089+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:32.115 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:32.112+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:32.309 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:32.301+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:32.311 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:32.304+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:32.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:32.323+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:32.334 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:32.325+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:32.830 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:32.824+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:32.869 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:32.866+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:33.095 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:33.088+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:33.107 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:33.103+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:33.307 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:33.294+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:33.308 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:33.296+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:33.359 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:33.353+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:33.362 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:33.358+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:33.808 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:33.801+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:33.890 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:33.887+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:34.094 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:34.087+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:34.137 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:34.134+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:34.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:34.250+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:34.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:34.253+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:34.366 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:34.360+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:34.367 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:34.362+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:34.853 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:34.848+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:34.927 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:34.924+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:35.055 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:35.049+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:35.182 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:35.179+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:35.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:35.300+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:35.306 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:35.302+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:35.342 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:35.335+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:35.343 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:35.338+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:35.832 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:35.826+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:35.971 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:35.968+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:36.028 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:36.022+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:36.140 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:36.136+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:36.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:36.306+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:36.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:36.308+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:36.349 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:36.343+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:36.350 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:36.345+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:36.844 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:36.837+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:36.924 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:36.921+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:37.066 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:37.060+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:37.113 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:37.108+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:37.318 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:37.313+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:37.321 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:37.317+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:37.339 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:37.333+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:37.340 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:37.337+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:37.874 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:37.867+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:37.882 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:37.879+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:38.113 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:38.106+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:38.151 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:38.147+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:38.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:38.319+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:38.329 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:38.321+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:38.330 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:38.324+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:38.332 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:38.328+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:38.841 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:38.835+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:38.894 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:38.888+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:39.073 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:39.066+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:39.181 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:39.176+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:39.306 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:39.302+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:39.306 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:39.304+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:39.348 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:39.342+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:39.351 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:39.346+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:39.866 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:39.859+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:39.874 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:39.868+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:40.075 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:40.069+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:40.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:40.219+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:40.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:40.280+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:40.285 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:40.282+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:40.318 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:40.311+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:40.319 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:40.314+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:40.865 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:40.859+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:40.886 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:40.882+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:41.049 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:41.042+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:41.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:41.235+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:41.257 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:41.238+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:41.258 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:41.239+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:41.274 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:41.268+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:41.275 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:41.270+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:41.906 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:41.901+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:41.935 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:41.930+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:42.023 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:42.015+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:42.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:42.206+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:42.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:42.207+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:42.230 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:42.227+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:42.253 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:42.247+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:42.254 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:42.249+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:42.913 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:42.907+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:42.923 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:42.919+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:43.014 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:43.008+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:43.189 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:43.183+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:43.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:43.186+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:43.222 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:43.216+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:43.223 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:43.218+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:43.233 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:43.229+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:43.901 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:43.897+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:43.940 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:43.934+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:43.973 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:43.968+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:44.178 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:44.173+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:44.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:44.176+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:44.192 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:44.186+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:44.193 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:44.189+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:44.239 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:44.234+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:44.876 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:44.872+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:44.982 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:44.976+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:44.986 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:44.980+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:45.224 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:45.218+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:45.225 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:45.221+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:45.230 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:45.226+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:45.231 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:45.228+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:45.277 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:45.274+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:45.850 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:45.847+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:45.945 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:45.940+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:45.960 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:45.954+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:46.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:46.225+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:46.231 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:46.227+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:46.244 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:46.239+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:46.245 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:46.242+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:46.251 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:46.246+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:46.828 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:46.825+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:46.948 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:46.942+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:46.986 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:46.980+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:47.190 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:47.184+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:47.190 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:47.187+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:47.207 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:47.204+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:47.208 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:47.205+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:47.217 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:47.214+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:47.874 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:47.868+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:47.952 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:47.946+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:47.987 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:47.981+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:48.174 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:48.167+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:48.175 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:48.170+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:48.201 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:48.194+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:48.201 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:48.196+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:48.244 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:48.238+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:48.884 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:48.879+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:48.928 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:48.921+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:49.039 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:49.034+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:49.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:49.148+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:49.153 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:49.150+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:49.198 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:49.192+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:49.200 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:49.195+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:49.209 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:49.205+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:49.936 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:49.930+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:49.972 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:49.967+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:50.006 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:49.998+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:50.111 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:50.104+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:50.112 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:50.107+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:50.223 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:50.218+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:50.233 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:50.227+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:50.233 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:50.229+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:50.903 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:50.899+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:50.929 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:50.923+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:51.004 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:50.998+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:51.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:51.117+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:51.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:51.120+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:51.222 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:51.217+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:51.225 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:51.221+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:51.253 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:51.248+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:51.861 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:18:51.857+0000 1694f700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:40.020643+0000 front 2019-12-08T21:17:37.133493+0000 (oldest deadline 2019-12-08T21:18:19.867134+0000) 2019-12-08T21:18:51.888 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:18:51.882+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.035238+0000 front 2019-12-08T21:17:39.037422+0000 (oldest deadline 2019-12-08T21:18:21.818145+0000) 2019-12-08T21:18:51.964 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:18:51.959+0000 1694f700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.055978+0000 front 2019-12-08T21:17:39.132368+0000 (oldest deadline 2019-12-08T21:18:22.176676+0000) 2019-12-08T21:18:52.139 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:52.134+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.051614+0000 front 2019-12-08T21:17:38.989279+0000 (oldest deadline 2019-12-08T21:18:20.469813+0000) 2019-12-08T21:18:52.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:52.136+0000 1694f700 -1 osd.0 438 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:52.194 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:52.188+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:39.145585+0000 front 2019-12-08T21:17:38.801403+0000 (oldest deadline 2019-12-08T21:18:24.810021+0000) 2019-12-08T21:18:52.194 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:52.190+0000 1694f700 -1 osd.7 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:52.215 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:18:52.211+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:17:37.373610+0000 front 2019-12-08T21:17:37.507996+0000 (oldest deadline 2019-12-08T21:18:22.670984+0000) 2019-12-08T21:18:52.489 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:18:52.481+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e438 v438) 2019-12-08T21:18:53.285 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:53.279+0000 1694f700 -1 osd.7 439 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:53.533 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:53.499+0000 1694f700 -1 osd.0 439 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:54.347 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:18:54.338+0000 1694f700 -1 osd.7 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119539 2.5s0 2:a1fac491:::9161132f-da48-40b4-a853-f341a1971d48.4507.619_8f96004800c543e895725e845064b3c4:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~128 in=128b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e433) 2019-12-08T21:18:54.550 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:18:54.499+0000 1694f700 -1 osd.0 440 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119557 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 1031133648 gen 2] snapc 0=[] ondisk+write+known_if_redirected e437) 2019-12-08T21:18:54.945 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:18:54.929+0000 20e25700 -1 RGWWatcher::handle_error cookie 1031133648 err (107) Transport endpoint is not connected 2019-12-08T21:20:51.386 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:51.380+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:51.728 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:51.717+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:51.836 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:51.830+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:52.355 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:52.348+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:52.533 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:52.528+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:52.769 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:52.761+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:52.860 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:52.853+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:53.349 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:53.342+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:53.429 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:53.426+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:53.521 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:53.517+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:53.823 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:53.817+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:53.864 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:53.857+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:54.016 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:20:54.012+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:20:54.317 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:54.311+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:54.389 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:54.386+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:54.571 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:54.563+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:54.800 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:54.793+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:54.874 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:54.868+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:55.018 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:20:55.012+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:20:55.361 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:55.358+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:20:55.361 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:55.356+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:55.362 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:55.359+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:55.537 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:55.534+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:55.827 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:55.819+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:55.912 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:55.904+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:56.030 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:20:56.026+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:20:56.343 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:56.336+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:56.396 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:56.392+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:20:56.397 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:56.394+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:56.575 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:56.571+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:56.869 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:56.857+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:56.893 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:56.886+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:57.048 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:20:57.045+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:20:57.394 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:57.389+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:57.437 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:57.434+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:20:57.438 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:57.436+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:57.556 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:57.552+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:57.833 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:57.822+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:57.871 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:57.864+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:58.014 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:20:58.011+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:20:58.424 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:58.414+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:58.441 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:58.437+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:20:58.446 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:58.444+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:58.541 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:58.535+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:58.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:58.828+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:20:58.845 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:58.839+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:59.054 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:20:59.051+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:20:59.402 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:59.398+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:20:59.403 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:20:59.400+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:20:59.469 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:20:59.463+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:20:59.500 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:20:59.496+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:20:59.809 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:20:59.803+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:20:59.852 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:20:59.846+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:00.019 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:00.016+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:00.365 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:00.362+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:00.366 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:00.363+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:00.475 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:00.469+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:00.536 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:00.532+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:00.806 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:00.800+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:00.828 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:00.822+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:01.013 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:01.010+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:01.381 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:01.376+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:01.382 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:01.378+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:01.509 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:01.503+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:01.538 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:01.534+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:01.801 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:01.794+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:01.805 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:01.799+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:02.056 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:02.052+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:02.407 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:02.402+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:02.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:02.405+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:02.480 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:02.474+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:02.515 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:02.508+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:02.757 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:02.751+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:02.803 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:02.797+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:03.033 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:03.029+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:03.428 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:03.422+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:03.428 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:03.425+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:03.528 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:03.522+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:03.537 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:03.533+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:03.787 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:03.780+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:03.809 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:03.804+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:04.025 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:04.021+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:04.399 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:04.394+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:04.403 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:04.399+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:04.501 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:04.497+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:04.502 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:04.481+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:04.749 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:04.742+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:04.798 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:04.792+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:05.058 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:05.054+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:05.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:05.404+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:05.409 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:05.406+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:05.483 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:05.477+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:05.492 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:05.486+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:05.718 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:05.712+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:05.851 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:05.842+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:06.096 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:06.092+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:06.448 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:06.442+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:06.449 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:06.444+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:06.498 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:06.492+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:06.537 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:06.532+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:06.704 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:06.698+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:06.835 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:06.829+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:07.121 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:07.117+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:07.448 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:07.442+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:07.448 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:07.444+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:07.492 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:07.486+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:07.544 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:07.539+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:07.719 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:07.713+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:07.831 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:07.824+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:08.124 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:08.119+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:08.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:08.405+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:08.413 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:08.408+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:08.504 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:08.500+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:08.535 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:08.529+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:08.740 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:08.733+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:08.843 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:08.837+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:09.151 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:09.146+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:09.430 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:09.424+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:09.430 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:09.426+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:09.491 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:09.485+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:09.534 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:09.527+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:09.742 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:09.736+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:09.889 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:09.882+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:10.109 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:10.104+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:10.455 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:10.451+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:10.464 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:10.460+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:10.466 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:10.463+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:10.586 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:10.579+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:10.782 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:10.770+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:10.922 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:10.916+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:10.938 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:10.929+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:11.154 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:11.149+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:11.464 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:11.459+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:11.470 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:11.464+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:11.481 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:11.471+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:11.581 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:11.575+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:11.797 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:11.791+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:11.920 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:11.913+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:11.921 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:11.916+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:12.162 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:12.157+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:12.434 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:12.429+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:12.435 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:12.431+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:12.479 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:12.476+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:12.604 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:12.598+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:12.758 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:12.753+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:12.940 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:12.933+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:12.940 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:12.936+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:13.181 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:13.173+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:13.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:13.447+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:13.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:13.448+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:13.482 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:13.477+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:13.637 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:13.630+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:13.769 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:13.763+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:13.898 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:13.892+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:13.899 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:13.894+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:14.202 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:14.198+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:14.460 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:14.456+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:14.462 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:14.459+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:14.472 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:14.468+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:14.652 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:14.646+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:14.806 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:14.799+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:14.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:14.866+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:14.877 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:14.871+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:15.218 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:15.215+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:15.431 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:15.427+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:15.432 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:15.429+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:15.453 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:15.448+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:15.686 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:15.680+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:15.855 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:15.849+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:15.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:15.870+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:15.884 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:15.874+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:16.253 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:16.248+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:16.455 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:16.449+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:16.478 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:16.473+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:16.478 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:16.475+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:16.702 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:16.696+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:16.867 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:16.861+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:16.870 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:16.866+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:16.886 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:16.879+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:17.244 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:17.240+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:17.450 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:17.446+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:17.483 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:17.479+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:17.484 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:17.481+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:17.659 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:17.653+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:17.852 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:17.846+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:17.875 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:17.868+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:17.876 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:17.871+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:18.263 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:18.259+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:18.434 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:18.430+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:18.474 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:18.471+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:18.476 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:18.473+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:18.692 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:18.687+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:18.840 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:18.834+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:18.883 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:18.878+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:18.889 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:18.882+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:19.226 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:19.223+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:19.450 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:19.446+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:19.453 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:19.450+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:19.457 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:19.454+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:19.717 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:19.712+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:19.865 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:19.859+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:19.865 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:19.861+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:19.890 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:19.884+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:20.202 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:20.198+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:20.438 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:20.433+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:20.441 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:20.438+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:20.486 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:20.481+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:20.704 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:20.699+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:20.876 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:20.870+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:20.901 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:20.895+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:20.902 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:20.898+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:21.236 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:21.231+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:21.397 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:21.393+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:21.398 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:21.395+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:21.441 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:21.436+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:21.742 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:21.732+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:21.927 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:21.923+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:21.928 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:21.923+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:21.931 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:21.927+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:22.208 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:22.204+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:22.356 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:22.351+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:22.357 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:22.354+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:22.485 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:22.481+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:22.699 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:22.695+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:22.895 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:22.889+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:22.895 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:22.892+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:22.964 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:22.959+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:23.021 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:21:23.019+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e462 v462) 2019-12-08T21:21:23.230 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:23.226+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:23.366 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:23.361+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:23.367 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:23.364+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:23.456 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:23.453+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:23.686 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:23.681+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:23.884 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:23.878+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:23.887 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:23.883+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:23.989 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:23.978+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:24.190 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:24.182+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:24.373 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:24.366+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:24.374 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:24.370+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:24.433 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:24.429+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:24.644 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:24.639+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:24.924 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:24.919+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:24.925 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:24.921+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:24.973 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:24.968+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:25.139 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:25.136+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:25.331 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:25.326+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:25.331 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:25.328+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:25.403 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:25.400+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:25.660 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:25.655+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:25.899 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:25.891+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:25.904 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:25.894+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:25.963 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:25.958+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:26.186 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:26.182+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:26.323 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:26.317+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:26.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:26.320+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:26.408 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:26.404+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:26.678 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:26.673+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:26.915 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:26.910+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:26.916 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:26.912+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:27.005 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:26.999+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:27.158 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:27.153+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:27.297 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:27.293+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:27.301 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:27.298+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:27.445 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:27.442+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:27.713 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:27.708+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:27.935 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:27.923+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:27.935 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:27.925+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:27.995 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:27.991+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:28.038 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:21:28.033+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e462 v462) 2019-12-08T21:21:28.185 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:28.182+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:28.309 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:28.302+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:28.309 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:28.304+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:28.475 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:28.472+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:28.680 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:28.674+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:28.936 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:28.930+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:28.936 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:28.932+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:29.034 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:29.029+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:29.167 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:29.163+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:29.347 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:29.342+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:29.347 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:29.345+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:29.521 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:29.518+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:29.648 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:29.640+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:29.901 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:29.885+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:29.902 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:29.888+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:30.079 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:30.074+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:30.143 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:30.140+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:30.336 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:30.330+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:30.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:30.334+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:30.525 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:30.520+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:30.668 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:30.662+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:30.873 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:30.868+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:30.874 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:30.869+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:31.078 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:31.072+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:31.100 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:31.096+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:31.303 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:31.298+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:31.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:31.301+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:31.512 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:31.507+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:31.623 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:31.618+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:31.836 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:31.830+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:31.837 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:31.832+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:32.110 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:32.105+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:32.160 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:32.150+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:32.345 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:32.341+0000 1694f700 -1 osd.0 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.824238+0000 front 2019-12-08T21:20:09.669301+0000 (oldest deadline 2019-12-08T21:20:54.927047+0000) 2019-12-08T21:21:32.346 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:32.343+0000 1694f700 -1 osd.0 462 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:32.535 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:21:32.531+0000 1694f700 -1 osd.3 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.974338+0000 front 2019-12-08T21:20:08.015680+0000 (oldest deadline 2019-12-08T21:20:52.329352+0000) 2019-12-08T21:21:32.661 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:21:32.650+0000 1694f700 -1 osd.7 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:07.766836+0000 front 2019-12-08T21:20:07.981596+0000 (oldest deadline 2019-12-08T21:20:50.547850+0000) 2019-12-08T21:21:32.817 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:32.809+0000 1694f700 -1 osd.5 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:09.289912+0000 front 2019-12-08T21:20:08.505008+0000 (oldest deadline 2019-12-08T21:20:51.308870+0000) 2019-12-08T21:21:32.821 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:32.813+0000 1694f700 -1 osd.5 462 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:33.072 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:21:33.066+0000 1694f700 -1 osd.4 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.165206+0000 front 2019-12-08T21:20:08.247385+0000 (oldest deadline 2019-12-08T21:20:51.588700+0000) 2019-12-08T21:21:33.087 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:21:33.082+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e462 v462) 2019-12-08T21:21:33.130 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:21:33.126+0000 1694f700 -1 osd.2 462 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:20:08.152731+0000 front 2019-12-08T21:20:08.159866+0000 (oldest deadline 2019-12-08T21:20:53.167591+0000) 2019-12-08T21:21:33.734 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:33.647+0000 1694f700 -1 osd.0 463 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:33.974 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:33.903+0000 1694f700 -1 osd.5 463 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:34.918 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:21:34.766+0000 1694f700 -1 osd.0 464 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:119838 5.5 5:a93a5511:::notify.2:head [watch ping cookie 5437401472 gen 1] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:35.012 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:21:35.004+0000 1694f700 -1 osd.5 464 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:119908 6.d 6:be7d4bd9:root::.bucket.meta.b41bfc3a3f8641cbb7e5e904621c91b8%3a9161132f-da48-40b4-a853-f341a1971d48.4507.621:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e462) 2019-12-08T21:21:35.480 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:21:35.464+0000 20e25700 -1 RGWWatcher::handle_error cookie 5437401472 err (107) Transport endpoint is not connected 2019-12-08T21:21:41.085 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:38:12.513 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.094 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:38:12.522 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.102 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:38:12.529 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.110 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:38:12.536 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.111 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:38:12.704 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.127 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:38:12.720 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.380 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:38:12.806 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.485 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:38:13.078 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:21:41.532 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:38:12.960 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:01.624 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:01.619+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:02.613 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:02.609+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:03.583 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:03.580+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:04.618 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:04.614+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:05.596 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:05.586+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:06.640 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:06.637+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:07.653 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:07.648+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:08.668 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:08.664+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:09.705 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:09.701+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:10.741 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:10.735+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:11.780 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:11.775+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:12.832 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:12.829+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:13.882 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:13.876+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:14.925 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:14.921+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:15.973 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:15.969+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:17.015 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:17.011+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:18.057 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:18.052+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:19.056 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:19.051+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:20.061 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:20.056+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:21.109 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:21.105+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:22.096 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:22.091+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:23.077 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:23.072+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:24.099 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:24.095+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:24.589 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:24.581+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:24.726 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:24.721+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:25.059 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:25.054+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:25.584 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:25.579+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:25.662 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:25.656+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:25.767 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:25.762+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:26.106 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:26.099+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:26.107 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:26.102+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:26.150 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:26.143+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:26.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:26.319+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:26.619 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:26.615+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:26.674 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:26.667+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:26.794 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:26.788+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:27.121 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:27.115+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:27.122 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:27.117+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:27.153 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:27.148+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:27.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:27.295+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:27.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:27.296+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:27.643 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:27.638+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:27.658 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:27.652+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:27.776 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:27.766+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:28.084 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:28.078+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:28.086 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:28.080+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:28.166 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:28.160+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:28.336 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:28.331+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:28.336 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:28.333+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:28.646 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:28.641+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:28.708 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:28.702+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:28.807 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:28.802+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:29.132 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:29.125+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:29.132 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:29.127+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:29.190 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:29.183+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:29.328 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:29.323+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:29.328 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:29.324+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:29.647 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:29.644+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:29.740 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:29.734+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:29.814 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:29.811+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:30.156 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:30.150+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:30.157 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:30.152+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:30.172 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:30.165+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:30.297 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:30.293+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:30.298 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:30.295+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:30.689 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:30.680+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:30.698 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:30.691+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:30.797 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:30.793+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:31.131 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:31.125+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:31.140 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:31.132+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:31.140 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:31.135+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:31.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:31.293+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:31.299 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:31.296+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:31.657 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:31.653+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:31.749 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:31.743+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:31.812 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:31.808+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:32.111 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:32.105+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:32.144 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:32.137+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:32.145 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:32.140+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:32.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:32.256+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:32.261 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:32.258+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:32.697 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:32.694+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:32.708 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:32.702+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:32.846 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:32.843+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:33.107 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:33.101+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:33.119 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:33.113+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:33.121 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:33.115+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:33.240 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:33.230+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:33.243 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:33.232+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:33.629 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:40:05.054 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.656 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:40:05.084 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.671 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:33.662+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:33.679 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:40:05.105 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.696 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:40:05.121 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.716 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:40:05.310 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.718 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:33.715+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:33.723 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:40:05.317 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.823 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:40:05.251 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.826 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:40:05.253 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.888 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:40:05.482 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:23:33.898 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:33.894+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:34.097 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:34.091+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:34.097 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:34.093+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:34.098 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:34.093+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:34.234 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:34.231+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:34.235 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:34.232+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:34.680 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:34.674+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:34.698 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:34.694+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:34.891 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:34.888+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:35.064 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:35.055+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:35.064 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:35.058+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:35.131 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:35.126+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:35.208 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:35.197+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:35.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:35.199+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:35.710 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:35.706+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:35.713 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:35.707+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:35.897 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:35.893+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:36.033 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:36.027+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:36.034 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:36.029+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:36.164 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:36.158+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:36.199 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:36.193+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:36.200 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:36.196+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:36.716 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:36.709+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:36.732 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:36.727+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:36.930 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:36.926+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:37.061 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:37.052+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:37.061 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:37.054+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:37.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:37.174+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:37.181 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:37.177+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:37.188 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:37.181+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:37.741 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:37.735+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:37.746 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:37.743+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:37.897 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:37.893+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:38.014 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:38.008+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:38.015 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:38.010+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:38.173 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:38.162+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:38.214 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:38.209+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:38.667 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:38.211+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:38.705 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:38.700+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:38.790 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:38.785+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:38.933 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:38.930+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:39.001 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:38.992+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:39.001 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:38.994+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:39.201 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:39.196+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:39.208 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:39.204+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:39.209 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:39.206+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:39.677 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:39.673+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:39.750 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:39.743+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:39.906 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:39.903+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:40.033 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:40.027+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:40.034 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:40.029+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:40.166 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:40.159+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:40.189 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:40.184+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:40.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:40.186+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:40.675 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:40.671+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:40.764 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:40.758+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:40.941 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:40.938+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:41.024 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:41.017+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:41.025 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:41.019+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:41.159 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:41.154+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:41.160 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:41.157+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:41.215 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:41.209+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:41.680 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:41.676+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:41.820 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:41.814+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:41.907 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:41.903+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:42.026 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:42.020+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:42.027 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:42.022+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:42.204 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:42.199+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:42.207 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:42.202+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:42.232 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:42.226+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:42.718 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:42.716+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:42.835 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:42.828+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:42.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:42.949+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:43.075 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:43.069+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:43.076 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:43.071+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:43.233 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:43.227+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:43.247 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:43.241+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:43.248 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:43.244+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:43.767 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:43.764+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:43.840 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:43.833+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:43.919 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:43.916+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:44.053 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:44.047+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:44.053 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:44.049+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:44.205 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:44.199+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:44.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:44.256+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:44.260 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:44.258+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:44.778 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:44.775+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:44.897 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:44.886+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:44.922 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:44.918+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:45.008 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:45.002+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:45.009 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:45.004+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:45.211 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:45.205+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:45.302 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:45.296+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:45.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:45.298+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:45.825 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:45.821+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:45.885 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:45.877+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:45.891 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:45.886+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:46.053 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:46.046+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:46.061 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:46.048+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:46.186 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:46.179+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:46.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:46.286+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:46.293 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:46.288+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:46.789 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:46.786+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:46.846 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:46.843+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:46.939 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:46.932+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:47.035 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:47.029+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:47.035 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:47.031+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:47.172 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:47.166+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:47.325 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:47.320+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:47.326 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:47.322+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:47.824 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:47.819+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:47.871 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:47.866+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:47.958 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:47.952+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:48.099 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:48.078+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:48.101 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:48.080+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:48.158 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:48.151+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:48.300 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:48.296+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:48.301 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:48.298+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:48.858 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:48.854+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:48.920 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:48.916+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:48.982 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:48.977+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:49.108 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:49.102+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:49.109 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:49.104+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:49.116 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:49.110+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:49.336 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:49.331+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:49.337 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:49.334+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:49.850 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:49.847+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:49.970 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:49.966+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:50.003 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:49.998+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:50.102 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:50.096+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:50.102 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:50.098+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:50.110 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:50.105+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:50.314 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:50.308+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:50.315 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:50.311+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:50.865 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:50.861+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:51.002 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:50.994+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:51.017 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:51.013+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:51.088 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:51.082+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:51.089 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:51.084+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:51.133 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:51.124+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:51.305 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:51.299+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:51.306 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:51.301+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:51.848 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:51.844+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:51.993 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:51.986+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:52.030 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:52.025+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:52.105 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:52.098+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:52.106 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:52.101+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:52.154 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:52.148+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:52.310 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:52.307+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:52.311 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:52.308+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:52.880 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:52.876+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:52.972 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:52.966+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:53.042 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:53.035+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:53.125 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:53.119+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:53.126 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:53.121+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:53.171 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:53.165+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:53.315 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:53.307+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:53.316 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:53.313+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:53.896 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:53.891+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:54.010 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:54.002+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:54.020 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:54.017+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:54.109 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:54.103+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:54.110 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:54.105+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:54.132 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:54.126+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:54.341 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:54.338+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:54.342 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:54.339+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:54.922 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:54.918+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:54.988 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:54.979+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:55.011 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:55.006+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:55.109 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:55.102+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:55.111 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:55.105+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:55.117 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:55.109+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:55.347 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:55.339+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:55.348 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:55.345+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:55.908 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:55.903+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:55.998 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:55.994+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:56.058 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:56.055+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:56.114 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:56.109+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:56.114 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:56.111+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:56.152 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:56.144+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:56.352 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:56.347+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:56.353 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:56.349+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:56.941 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:56.930+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:56.958 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:56.955+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:57.043 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:57.038+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:57.093 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:57.087+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:57.093 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:57.089+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:57.111 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:57.106+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:57.339 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:57.333+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:57.340 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:57.337+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:57.958 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:57.954+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:57.998 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:57.993+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:58.068 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:58.064+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:58.074 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:58.069+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:58.075 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:58.071+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:58.087 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:58.082+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:58.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:58.320+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:58.324 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:58.322+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:23:58.686 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:23:58.682+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e471 v471) 2019-12-08T21:23:58.978 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:23:58.974+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:23:59.002 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:23:58.999+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:23:59.044 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:59.038+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:23:59.044 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:23:59.041+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:23:59.045 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:23:59.041+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:23:59.059 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:23:59.054+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:23:59.358 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:59.352+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:23:59.359 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:23:59.356+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:00.012 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:24:00.009+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:24:00.032 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:00.026+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:24:00.032 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:00.028+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:24:00.037 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:24:00.034+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:24:00.039 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:24:00.034+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:24:00.086 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:24:00.080+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:24:00.340 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:00.336+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:24:00.341 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:00.337+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:00.989 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:24:00.983+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:24:01.033 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:24:01.027+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:24:01.035 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:24:01.029+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:24:01.036 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:01.028+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:24:01.036 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:01.031+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:24:01.125 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:24:01.120+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:24:01.346 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:01.342+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:24:01.347 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:01.344+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:01.988 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:24:01.982+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:24:01.990 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:24:01.986+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:24:01.991 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:24:01.986+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:24:02.018 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:02.013+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:24:02.019 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:02.015+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:24:02.102 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:24:02.096+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:24:02.387 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:02.382+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:24:02.388 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:02.384+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:02.947 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:24:02.942+0000 1694f700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:42.694634+0000 front 2019-12-08T21:22:42.337312+0000 (oldest deadline 2019-12-08T21:23:23.981573+0000) 2019-12-08T21:24:02.953 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:24:02.949+0000 1694f700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.437670+0000 front 2019-12-08T21:22:39.444361+0000 (oldest deadline 2019-12-08T21:23:24.432706+0000) 2019-12-08T21:24:02.972 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:02.967+0000 1694f700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.632730+0000 front 2019-12-08T21:22:40.453115+0000 (oldest deadline 2019-12-08T21:23:25.711481+0000) 2019-12-08T21:24:02.973 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:02.969+0000 1694f700 -1 osd.5 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:24:03.007 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:24:03.001+0000 1694f700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:40.629420+0000 front 2019-12-08T21:22:40.626804+0000 (oldest deadline 2019-12-08T21:23:26.000211+0000) 2019-12-08T21:24:03.059 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:24:03.053+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:24:03.436 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:03.432+0000 1694f700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:39.977246+0000 front 2019-12-08T21:22:40.084384+0000 (oldest deadline 2019-12-08T21:23:25.789865+0000) 2019-12-08T21:24:03.437 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:03.434+0000 1694f700 -1 osd.0 471 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:03.774 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:24:03.748+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 44sec e471 v471) 2019-12-08T21:24:03.963 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:03.944+0000 1694f700 -1 osd.5 472 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:24:04.090 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:24:04.086+0000 1694f700 -1 osd.4 471 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:22:41.222957+0000 front 2019-12-08T21:22:41.300034+0000 (oldest deadline 2019-12-08T21:23:24.697645+0000) 2019-12-08T21:24:04.521 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:04.401+0000 1694f700 -1 osd.0 472 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:04.954 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:24:04.951+0000 1694f700 -1 osd.5 472 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120094 6.d 6.9bd2be7d (undecoded) ondisk+read+known_if_redirected e471) 2019-12-08T21:24:05.623 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:24:05.485+0000 1694f700 -1 osd.0 473 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120136 5.5 5:a93a5511:::notify.2:head [watch ping cookie 416127424 gen 1] snapc 0=[] ondisk+write+known_if_redirected e471) 2019-12-08T21:24:06.273 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:24:06.259+0000 20e25700 -1 RGWWatcher::handle_error cookie 416127424 err (107) Transport endpoint is not connected 2019-12-08T21:24:08.912 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainer) ... ok 2019-12-08T21:24:47.122 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:24:47.116+0000 30f85700 -1 osd.1 476 build_incremental_map_msg missing incremental map 476 2019-12-08T21:24:47.252 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:24:47.173+0000 114ae700 -1 osd.1 476 build_incremental_map_msg missing incremental map 476 2019-12-08T21:24:47.256 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:24:47.184+0000 3578e700 -1 osd.1 476 build_incremental_map_msg missing incremental map 476 2019-12-08T21:24:47.284 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:24:47.240+0000 31786700 -1 osd.1 476 build_incremental_map_msg missing incremental map 476 2019-12-08T21:24:47.399 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:24:47.320+0000 31786700 -1 osd.1 476 build_incremental_map_msg missing incremental map 476 2019-12-08T21:24:47.578 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:24:47.505+0000 37792700 -1 osd.1 476 build_incremental_map_msg missing incremental map 476 2019-12-08T21:24:54.009 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerFileList (test.functional.tests.TestContainer) ... ok 2019-12-08T21:24:55.848 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-08T21:24:59.269 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainer) ... ok 2019-12-08T21:24:59.788 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerInfo (test.functional.tests.TestContainer) ... ok 2019-12-08T21:25:00.771 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-08T21:25:31.182 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:42:02.610 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.188 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:42:02.616 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.198 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:42:02.623 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.201 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:42:02.628 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.203 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:42:02.796 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.217 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:42:02.811 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.390 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:42:02.818 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.405 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:42:02.832 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:31.503 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:42:03.096 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:25:38.577 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:38.573+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:39.608 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:39.604+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:40.570 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:40.567+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:41.595 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:41.592+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:42.557 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:42.553+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:43.574 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:43.570+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:44.543 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:44.539+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:45.596 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:45.593+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:46.639 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:46.636+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:47.681 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:47.677+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:48.650 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:48.645+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:48.651 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:48.647+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:49.701 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:49.696+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:49.702 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:49.698+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:50.078 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:50.074+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:50.736 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:50.730+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:50.736 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:50.732+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:50.938 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:50.935+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:51.064 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:51.060+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:51.098 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:51.091+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:51.339 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:51.335+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:51.753 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:51.747+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:51.754 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:51.750+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:51.930 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:51.927+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:51.938 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:51.934+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:52.098 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:52.093+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:52.104 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:52.100+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:52.392 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:52.386+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:52.756 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:52.750+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:52.757 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:52.752+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:52.927 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:52.924+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:52.959 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:52.956+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:53.067 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:53.061+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:53.144 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:53.138+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:53.362 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:53.354+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:53.770 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:53.764+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:53.771 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:53.767+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:53.954 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:53.951+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:53.957 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:53.954+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:54.109 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:54.104+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:54.112 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:54.107+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:54.377 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:54.371+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:54.783 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:54.777+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:54.784 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:54.780+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:54.922 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:54.919+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:55.002 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:54.998+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:55.116 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:55.110+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:55.118 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:55.112+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:55.432 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:55.425+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:55.780 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:55.774+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:55.781 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:55.776+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:55.901 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:55.896+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:55.989 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:55.985+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:56.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:56.072+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:56.077 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:56.075+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:25:56.115 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:56.109+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:56.456 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:56.450+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:56.808 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:56.798+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:56.809 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:56.801+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:56.925 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:56.922+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:56.979 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:56.974+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:57.096 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:57.090+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:57.129 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:57.124+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:57.130 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:57.127+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:25:57.451 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:57.445+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:57.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:57.849+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:57.861 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:57.851+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:57.958 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:57.952+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:58.005 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:58.002+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:58.122 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:58.116+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:58.180 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:58.175+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:58.181 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:58.177+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:25:58.491 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:58.485+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:58.875 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:58.870+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:58.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:58.874+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:58.990 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:25:58.985+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:25:58.995 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:58.990+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:25:59.113 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:25:59.107+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:25:59.163 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:59.159+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:25:59.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:25:59.161+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:25:59.459 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:25:59.453+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:25:59.862 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:59.856+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:25:59.863 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:25:59.858+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:25:59.986 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:25:59.982+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:00.006 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:00.003+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:00.074 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:00.068+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:00.171 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:00.168+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:00.172 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:00.169+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:00.501 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:00.493+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:00.910 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:00.903+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:00.911 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:00.906+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:00.977 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:00.972+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:00.982 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:00.977+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:01.132 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:01.122+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:01.155 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:01.149+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:01.155 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:01.150+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:01.518 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:01.512+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:01.869 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:01.860+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:01.869 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:01.864+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:02.015 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:02.010+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:02.016 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:02.011+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:02.128 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:02.122+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:02.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:02.162+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:02.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:02.163+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:02.541 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:02.534+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:02.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:02.828+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:02.840 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:02.833+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:03.014 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:03.011+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:03.021 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:03.017+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:03.096 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:03.089+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:03.196 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:03.190+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:03.196 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:03.192+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:03.522 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:03.516+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:03.857 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:03.850+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:03.857 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:03.852+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:04.012 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:04.004+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:04.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:04.031+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:04.081 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:04.072+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:04.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:04.155+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:04.164 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:04.157+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:04.558 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:04.553+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:04.829 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:04.822+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:04.829 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:04.825+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:04.996 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:04.993+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:04.999 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:04.995+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:05.039 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:05.033+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:05.118 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:05.114+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:05.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:05.119+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:05.570 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:05.565+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:05.851 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:05.844+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:05.855 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:05.847+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:05.989 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:05.985+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:06.021 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:06.016+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:06.043 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:06.037+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:06.085 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:06.079+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:06.086 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:06.082+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:06.566 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:06.559+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:06.882 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:06.876+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:06.882 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:06.878+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:07.021 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:07.018+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:07.035 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:07.026+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:07.062 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:07.057+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:07.097 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:07.093+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:07.098 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:07.095+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:07.571 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:07.560+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:07.862 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:07.856+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:07.863 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:07.858+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:08.042 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:08.038+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:08.046 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:08.042+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:08.089 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:08.083+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:08.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:08.134+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:08.141 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:08.137+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:08.549 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:08.543+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:08.871 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:08.865+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:08.872 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:08.867+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:09.078 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:09.072+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:09.081 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:09.078+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:09.089 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:09.085+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:09.136 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:09.132+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:09.136 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:09.133+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:09.512 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:09.505+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:09.857 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:09.852+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:09.859 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:09.856+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:10.058 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:10.052+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:10.076 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:10.072+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:10.100 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:10.092+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:10.148 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:10.139+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:10.148 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:10.142+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:10.502 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:10.497+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:10.824 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:10.818+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:10.827 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:10.823+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:11.049 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:11.041+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:11.058 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:11.054+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:11.088 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:11.083+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:11.192 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:11.189+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:11.193 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:11.191+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:11.548 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:11.541+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:11.805 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:11.799+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:11.808 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:11.804+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:12.063 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:12.056+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:12.104 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:12.101+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:12.141 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:12.136+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:12.228 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:12.223+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:12.229 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:12.226+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:12.566 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:12.560+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:12.836 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:12.829+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:12.840 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:12.834+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:13.021 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:13.015+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:13.089 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:13.086+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:13.132 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:13.129+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:13.246 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:13.235+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:13.246 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:13.237+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:13.575 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:13.569+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:13.808 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:13.800+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:13.809 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:13.803+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:14.014 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:14.009+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:14.108 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:14.105+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:14.109 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:14.105+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:14.219 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:14.212+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:14.222 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:14.219+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:14.549 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:14.543+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:14.839 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:14.834+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:14.843 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:14.838+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:14.978 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:14.972+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:15.070 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:15.066+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:15.071 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:15.066+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:15.190 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:15.185+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:15.191 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:15.188+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:15.572 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:15.566+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:15.838 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:15.832+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:15.839 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:15.834+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:15.967 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:15.961+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:16.040 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:16.036+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:16.048 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:16.044+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:16.169 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:16.164+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:16.170 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:16.165+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:16.589 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:16.583+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:16.812 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:16.806+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:16.813 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:16.808+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:16.933 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:16.927+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:17.000 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:16.996+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:17.015 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:17.009+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:17.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:17.144+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:17.148 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:17.145+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:17.575 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:17.571+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:17.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:17.828+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:17.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:17.830+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:17.985 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:17.980+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:17.991 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:17.987+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:18.022 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:18.018+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:18.120 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:18.115+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:18.122 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:18.118+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:18.616 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:18.610+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:18.831 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:18.826+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:18.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:18.830+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:18.960 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:18.949+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:18.989 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:18.985+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:19.021 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:19.018+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:19.098 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:19.093+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:19.099 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:19.096+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:19.667 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:19.661+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:19.878 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:19.872+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:19.881 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:19.876+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:19.958 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:19.954+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:19.976 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:19.970+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:20.009 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:20.002+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:20.078 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:20.075+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:20.080 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:20.078+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:20.624 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:20.619+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:20.884 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:20.878+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:20.885 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:20.880+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:20.950 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:20.948+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:21.001 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:20.996+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:21.017 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:21.011+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:21.087 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:21.083+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:21.090 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:21.087+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:21.623 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:21.617+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:21.896 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:21.889+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:21.896 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:21.892+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:21.969 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:21.965+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:22.044 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:22.039+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:22.056 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:22.051+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:22.081 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:22.077+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:22.084 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:22.081+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:22.618 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:22.613+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:22.866 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:22.853+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:22.868 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:22.862+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:22.950 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:22.946+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:23.058 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:23.055+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:23.060 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:23.055+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:23.060 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:23.057+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:23.069 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:23.064+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:23.601 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:23.595+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:23.843 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:23.836+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:23.843 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:23.839+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:23.963 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:23.955+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:24.082 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:24.077+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:24.098 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:24.094+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:24.102 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:24.098+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:24.121 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:24.115+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:24.199 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:26:24.196+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e483 v483) 2019-12-08T21:26:24.576 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:24.570+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:24.893 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:24.885+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:24.896 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:24.887+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:24.963 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:24.958+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:25.090 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:25.085+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:25.092 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:25.086+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:25.131 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:25.126+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:25.149 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:25.142+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:25.569 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:25.564+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:25.915 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:25.909+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:25.916 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:25.911+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:26.014 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:26.009+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:26.095 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:26.090+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:26.096 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:26.091+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:26.098 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:26.094+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:26.106 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:26.100+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:26.615 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:26.608+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:26.908 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:26.899+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:26.908 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:26.902+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:27.024 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:27.020+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:27.078 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:27.074+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:27.115 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:27.109+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:27.136 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:27.131+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:27.137 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:27.133+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:27.593 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:27.586+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:27.904 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:27.899+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:27.907 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:27.903+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:28.019 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:28.014+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:28.057 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:28.052+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:28.146 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:28.140+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:28.147 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:28.142+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:28.154 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:28.146+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:28.639 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:26:28.633+0000 1694f700 -1 osd.4 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.072808+0000 front 2019-12-08T21:25:05.270626+0000 (oldest deadline 2019-12-08T21:25:50.938669+0000) 2019-12-08T21:26:28.872 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:28.865+0000 1694f700 -1 osd.5 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.348152+0000 front 2019-12-08T21:25:07.349355+0000 (oldest deadline 2019-12-08T21:25:48.411675+0000) 2019-12-08T21:26:28.872 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:28.867+0000 1694f700 -1 osd.5 483 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:29.026 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:26:29.021+0000 1694f700 -1 osd.3 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:06.571427+0000 front 2019-12-08T21:25:06.558259+0000 (oldest deadline 2019-12-08T21:25:51.248865+0000) 2019-12-08T21:26:29.061 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:26:29.057+0000 1694f700 -1 osd.2 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:05.807839+0000 front 2019-12-08T21:25:06.054216+0000 (oldest deadline 2019-12-08T21:25:50.436680+0000) 2019-12-08T21:26:29.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:29.115+0000 1694f700 -1 osd.0 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:07.437055+0000 front 2019-12-08T21:25:07.498747+0000 (oldest deadline 2019-12-08T21:25:50.325040+0000) 2019-12-08T21:26:29.121 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:29.117+0000 1694f700 -1 osd.0 483 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:29.184 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:26:29.178+0000 1694f700 -1 osd.7 483 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:25:04.213893+0000 front 2019-12-08T21:25:04.198735+0000 (oldest deadline 2019-12-08T21:25:49.403626+0000) 2019-12-08T21:26:29.259 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:26:29.251+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 45sec e483 v483) 2019-12-08T21:26:30.173 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:26:30.140+0000 1694f700 -1 osd.5 484 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120417 6.d 6.cc86464d (undecoded) ondisk+read+known_if_redirected e483) 2019-12-08T21:26:30.202 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:30.194+0000 1694f700 -1 osd.0 484 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:31.184 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:26:31.177+0000 1694f700 -1 osd.0 485 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120443 5.5 5:a93a5511:::notify.2:head [watch ping cookie 363656064 gen 1] snapc 0=[] ondisk+write+known_if_redirected e483) 2019-12-08T21:26:31.784 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:26:31.772+0000 20e25700 -1 RGWWatcher::handle_error cookie 363656064 err (107) Transport endpoint is not connected 2019-12-08T21:26:52.411 INFO:teuthology.orchestra.run.smithi129.stderr:testContainerNameLimit (test.functional.tests.TestContainer) ... ok 2019-12-08T21:27:07.130 INFO:teuthology.orchestra.run.smithi129.stderr:testCreate (test.functional.tests.TestContainer) ... ok 2019-12-08T21:27:12.993 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:12.948+0000 30f85700 -1 osd.1 488 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:13.008 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:12.965+0000 30f85700 -1 osd.1 488 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:13.219 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:13.079+0000 11caf700 -1 osd.1 488 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:13.438 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:13.431+0000 34f8d700 -1 osd.1 488 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:14.105 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:13.972+0000 3578e700 -1 osd.1 488 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:15.757 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:15.703+0000 3378a700 -1 osd.1 490 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:17.421 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:27:17.153+0000 37792700 -1 osd.1 490 build_incremental_map_msg missing incremental map 488 2019-12-08T21:27:42.690 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:42.687+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:43.590 INFO:teuthology.orchestra.run.smithi129.stderr:testDelete (test.functional.tests.TestContainer) ... ok 2019-12-08T21:27:43.652 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:43.649+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:44.515 INFO:teuthology.orchestra.run.smithi129.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-08T21:27:44.656 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:44.653+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:45.689 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:45.686+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:46.659 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:46.656+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:47.654 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:47.651+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:48.686 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:48.684+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:49.710 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:49.708+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:50.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:50.690+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:51.524 INFO:teuthology.orchestra.run.smithi129.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainer) ... ok 2019-12-08T21:27:51.701 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:51.699+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:52.113 INFO:teuthology.orchestra.run.smithi129.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-08T21:27:52.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:52.692+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:53.658 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:53.655+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:54.669 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:54.667+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:55.643 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:55.640+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:56.670 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:56.666+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:57.658 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:57.654+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:58.695 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:58.691+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:27:59.721 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:27:59.717+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:00.715 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:00.711+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:01.697 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:01.693+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:02.668 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:02.664+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:03.670 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:03.667+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:04.705 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:04.702+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:05.757 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:05.755+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:06.737 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:06.735+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:07.762 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:07.760+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:08.786 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:08.782+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:09.784 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:09.782+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:10.809 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:10.806+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:11.766 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:11.763+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:12.727 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:12.721+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:13.736 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:13.733+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:14.742 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:14.738+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:14.742 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:14.740+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:15.725 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:15.720+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:15.727 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:15.725+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:16.076 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:16.070+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:16.255 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:16.250+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:16.701 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:16.698+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:16.703 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:16.701+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:16.936 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:16.930+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:17.117 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:17.111+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:17.266 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:17.263+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:17.730 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:17.723+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:17.731 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:17.726+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:17.949 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:17.944+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:18.076 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:18.071+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:18.303 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:18.299+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:18.685 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:18.681+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:18.722 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:18.713+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:18.723 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:18.716+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:18.970 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:18.965+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:19.056 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:19.051+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:19.299 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:19.295+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:19.671 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:19.667+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:19.706 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:19.701+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:19.707 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:19.703+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:19.949 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:19.945+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:20.063 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:20.057+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:20.175 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:20.170+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:20.293 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:20.290+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:20.643 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:20.640+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:20.687 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:20.683+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:20.690 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:20.687+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:20.954 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:20.947+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:21.084 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:21.078+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:21.182 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:21.176+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:21.265 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:21.262+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:21.678 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:21.673+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:21.715 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:21.711+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:21.717 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:21.715+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:21.952 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:21.947+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:22.069 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:22.064+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:22.203 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:22.198+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:22.293 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:22.290+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:22.710 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:22.704+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:22.770 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:22.764+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:22.770 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:22.767+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:22.990 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:22.985+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:23.026 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:23.019+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:23.248 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:23.243+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:23.312 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:23.309+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:23.697 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:23.691+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:23.721 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:23.718+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:23.723 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:23.719+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:23.984 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:23.979+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:24.062 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:24.054+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:24.278 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:24.273+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:24.332 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:24.328+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:24.664 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:24.660+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:24.710 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:24.707+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:24.711 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:24.708+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:24.961 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:44:56.388 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:24.974 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:44:56.403 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:24.975 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:44:56.402 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:24.977 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:44:56.406 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:25.015 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:25.010+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:25.028 INFO:tasks.ceph.osd.5.smithi181.stderr:2019-12-08T21:28:25.023+0000 1694f700 -1 osd.5 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.405819+0000 front 2019-12-08T21:27:32.489592+0000 (oldest deadline 2019-12-08T21:28:15.215780+0000) 2019-12-08T21:28:25.029 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:44:56.623 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:25.037 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:44:56.630 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:25.193 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:44:56.619 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:25.312 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:25.309+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:25.320 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:25.314+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:25.354 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:44:56.782 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:25.369 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:44:56.962 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:28:25.629 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:25.625+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:25.713 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:25.709+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:25.714 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:25.711+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:26.053 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:26.049+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:26.319 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:26.315+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:26.326 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:26.323+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:26.604 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:26.601+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:26.692 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:26.689+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:26.693 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:26.690+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:27.087 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:27.080+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:27.356 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:27.353+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:27.358 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:27.348+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:27.625 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:27.621+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:27.743 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:27.739+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:27.744 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:27.741+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:28.132 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:28.127+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:28.346 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:28.342+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:28.367 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:28.359+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:28.590 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:28.586+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:28.701 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:28.697+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:28.704 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:28.702+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:29.113 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:29.107+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:29.314 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:29.311+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:29.321 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:29.315+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:29.604 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:29.600+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:29.731 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:29.727+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:29.735 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:29.732+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:30.120 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:30.114+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:30.269 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:30.266+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:30.342 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:30.336+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:30.613 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:30.609+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:30.781 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:30.776+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:30.781 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:30.778+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:31.154 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:31.148+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:31.280 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:31.276+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:31.303 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:31.297+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:31.658 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:31.655+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:31.824 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:31.819+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:31.825 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:31.822+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:32.167 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:32.161+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:32.283 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:32.276+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:32.301 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:32.294+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:32.705 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:32.702+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:32.811 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:32.808+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:32.813 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:32.811+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:33.172 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:33.168+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:33.300 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:33.292+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:33.331 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:33.327+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:33.720 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:33.717+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:33.852 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:33.849+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:33.854 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:33.850+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:34.203 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:34.190+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:34.266 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:34.261+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:34.364 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:34.361+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:34.767 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:34.762+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:34.878 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:34.872+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:34.879 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:34.875+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:35.202 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:35.197+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:35.237 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:35.231+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:35.342 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:35.339+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:35.804 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:35.801+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:35.836 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:35.833+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:35.837 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:35.835+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:36.177 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:36.171+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:36.285 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:36.279+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:36.331 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:36.327+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:36.801 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:36.796+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:36.802 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:36.799+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:36.818 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:36.814+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:37.139 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:37.133+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:37.254 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:37.248+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:37.337 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:37.333+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:37.811 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:37.807+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:37.815 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:37.810+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:37.816 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:37.813+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:38.150 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:38.144+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:38.290 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:38.284+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:38.361 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:38.357+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:38.836 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:38.831+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:38.851 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:38.846+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:38.851 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:38.849+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:39.162 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:39.156+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:39.295 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:39.289+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:39.350 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:39.346+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:39.873 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:39.868+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:39.892 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:39.886+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:39.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:39.890+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:40.192 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:40.187+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:40.322 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:40.316+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:40.345 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:40.341+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:40.873 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:40.869+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:40.880 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:40.876+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:40.882 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:40.878+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:41.218 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:41.212+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:41.373 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:41.367+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:41.374 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:41.371+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:41.828 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:41.822+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:41.870 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:41.864+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:41.874 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:41.866+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:42.229 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:42.224+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:42.334 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:42.328+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:42.376 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:42.371+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:42.811 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:42.805+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:42.900 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:42.893+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:42.900 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:42.896+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:43.264 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:43.258+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:43.362 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:43.356+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:43.380 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:43.374+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:43.785 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:43.779+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:43.919 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:43.913+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:43.920 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:43.915+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:44.316 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:44.310+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:44.369 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:44.361+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:44.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:44.393+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:44.773 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:44.768+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:44.964 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:44.959+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:44.965 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:44.961+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:45.328 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:45.322+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:45.346 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:45.340+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:45.432 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:45.428+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:45.782 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:45.775+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:45.994 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:45.985+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:46.001 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:45.992+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:46.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:46.308+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:46.336 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:46.330+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:46.409 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:46.403+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:46.823 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:46.818+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:46.972 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:46.961+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:46.973 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:46.964+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:47.359 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:47.351+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:47.364 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:47.358+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:47.406 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:47.401+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:47.845 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:47.839+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:47.944 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:47.939+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:47.945 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:47.941+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:48.368 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:48.362+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:48.394 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:48.388+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:48.413 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:48.407+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:48.827 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:48.823+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:48.951 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:48.944+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:48.952 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:48.947+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:49.371 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:49.364+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:49.375 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:49.369+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:49.419 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:49.414+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:49.876 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:49.871+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:49.939 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:49.935+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:49.944 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:49.938+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:49.961 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:28:49.955+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e491 v491) 2019-12-08T21:28:50.347 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:50.342+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:50.388 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:50.382+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:50.398 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:50.393+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:50.908 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:50.904+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:50.916 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:50.913+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:50.920 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:50.917+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:51.337 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:51.332+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:51.372 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:51.369+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:51.373 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:51.366+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:51.923 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:51.919+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:51.924 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:51.921+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:51.943 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:51.939+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:52.343 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:52.338+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:52.345 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:52.339+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:52.393 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:52.390+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:52.886 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:52.881+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:52.890 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:52.888+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:52.930 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:52.922+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:53.353 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:53.347+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:53.356 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:53.349+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:53.431 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:53.427+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:53.923 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:53.920+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:53.927 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:53.914+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:53.931 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:53.925+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:54.332 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:54.326+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:54.399 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:54.386+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:54.420 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:54.417+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:54.885 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:54.877+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:54.885 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:54.881+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:54.939 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:54.936+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:54.974 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:28:54.970+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e491 v491) 2019-12-08T21:28:55.358 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:55.353+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:55.439 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:55.434+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:55.442 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:55.439+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:55.893 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:55.886+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:55.894 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:55.890+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:55.985 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:55.981+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:56.338 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:56.331+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:56.417 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:56.413+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:56.425 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:56.418+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:56.944 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:56.940+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:56.948 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:56.944+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:56.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:56.949+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:57.348 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:57.342+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:57.404 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:57.398+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:57.439 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:57.435+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:57.934 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:57.929+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:57.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:57.977+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:57.984 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:57.981+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:58.314 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:58.308+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:58.371 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:58.365+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:58.415 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:58.410+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:58.974 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:58.970+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:28:58.999 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:58.994+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:59.000 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:58.997+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:59.338 INFO:tasks.ceph.osd.4.smithi181.stderr:2019-12-08T21:28:59.332+0000 1694f700 -1 osd.4 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.042936+0000 front 2019-12-08T21:27:34.042372+0000 (oldest deadline 2019-12-08T21:28:19.916878+0000) 2019-12-08T21:28:59.373 INFO:tasks.ceph.osd.7.smithi181.stderr:2019-12-08T21:28:59.368+0000 1694f700 -1 osd.7 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:34.034946+0000 front 2019-12-08T21:27:33.951616+0000 (oldest deadline 2019-12-08T21:28:16.678673+0000) 2019-12-08T21:28:59.465 INFO:tasks.ceph.osd.2.smithi129.stderr:2019-12-08T21:28:59.461+0000 1694f700 -1 osd.2 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.616043+0000 front 2019-12-08T21:27:32.703332+0000 (oldest deadline 2019-12-08T21:28:15.454129+0000) 2019-12-08T21:28:59.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:59.966+0000 1694f700 -1 osd.0 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.282047+0000 front 2019-12-08T21:27:32.297322+0000 (oldest deadline 2019-12-08T21:28:14.466457+0000) 2019-12-08T21:28:59.981 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:28:59.974+0000 1694f700 -1 osd.0 491 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:28:59.981 INFO:tasks.ceph.osd.3.smithi129.stderr:2019-12-08T21:28:59.969+0000 1694f700 -1 osd.3 491 heartbeat_check: no reply from 172.21.15.129:6820 osd.1 since back 2019-12-08T21:27:32.557978+0000 front 2019-12-08T21:27:32.571901+0000 (oldest deadline 2019-12-08T21:28:18.310725+0000) 2019-12-08T21:29:00.043 INFO:tasks.ceph.mon.a.smithi129.stderr:2019-12-08T21:29:00.040+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.1 [v2:172.21.15.129:6812/10197,v1:172.21.15.129:6813/10197] for 43sec e491 v491) 2019-12-08T21:29:00.969 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:29:00.946+0000 1694f700 -1 osd.0 492 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:29:01.922 INFO:tasks.ceph.osd.0.smithi129.stderr:2019-12-08T21:29:01.915+0000 1694f700 -1 osd.0 493 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4530.0:120668 5.5 5:a93a5511:::notify.2:head [watch reconnect cookie 821424176 gen 1] snapc 0=[] ondisk+write+known_if_redirected e487) 2019-12-08T21:29:02.643 INFO:tasks.rgw.client.0.smithi129.stdout:2019-12-08T21:29:02.636+0000 20e25700 -1 RGWWatcher::handle_error cookie 821424176 err (107) Transport endpoint is not connected 2019-12-08T21:29:17.252 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:45:48.679 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.257 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:45:48.685 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.268 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:45:48.694 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.271 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:45:48.697 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.284 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:45:48.877 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.289 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:45:48.882 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.339 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:45:48.767 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.354 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:45:48.781 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:17.482 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:45:49.076 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:29:33.022 INFO:tasks.ceph.osd.1.smithi129.stderr:*** Caught signal (Segmentation fault) ** 2019-12-08T21:29:33.023 INFO:tasks.ceph.osd.1.smithi129.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:29:33.085 INFO:tasks.ceph.osd.1.smithi129.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:29:33.085 INFO:tasks.ceph.osd.1.smithi129.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:29:33.085 INFO:tasks.ceph.osd.1.smithi129.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:29:33.085 INFO:tasks.ceph.osd.1.smithi129.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:29:33.085 INFO:tasks.ceph.osd.1.smithi129.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:29:33.085 INFO:tasks.ceph.osd.1.smithi129.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:29:33.086 INFO:tasks.ceph.osd.1.smithi129.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:29:33.087 INFO:tasks.ceph.osd.1.smithi129.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:29:33.107 INFO:tasks.ceph.osd.1.smithi129.stderr:2019-12-08T21:29:33.102+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-08T21:29:33.107 INFO:tasks.ceph.osd.1.smithi129.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:29:33.107 INFO:tasks.ceph.osd.1.smithi129.stderr: 2019-12-08T21:29:33.108 INFO:tasks.ceph.osd.1.smithi129.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:29:33.108 INFO:tasks.ceph.osd.1.smithi129.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:29:33.108 INFO:tasks.ceph.osd.1.smithi129.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:29:33.108 INFO:tasks.ceph.osd.1.smithi129.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:29:33.108 INFO:tasks.ceph.osd.1.smithi129.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:29:33.108 INFO:tasks.ceph.osd.1.smithi129.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:29:33.109 INFO:tasks.ceph.osd.1.smithi129.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:29:33.110 INFO:tasks.ceph.osd.1.smithi129.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-08T21:29:33.110 INFO:tasks.ceph.osd.1.smithi129.stderr: 2019-12-08T21:29:33.577 INFO:tasks.ceph.osd.1.smithi129.stderr: 0> 2019-12-08T21:29:33.102+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-08T21:29:33.578 INFO:tasks.ceph.osd.1.smithi129.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:29:33.578 INFO:tasks.ceph.osd.1.smithi129.stderr: 2019-12-08T21:29:33.578 INFO:tasks.ceph.osd.1.smithi129.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:29:33.578 INFO:tasks.ceph.osd.1.smithi129.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:29:33.578 INFO:tasks.ceph.osd.1.smithi129.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:29:33.578 INFO:tasks.ceph.osd.1.smithi129.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:29:33.579 INFO:tasks.ceph.osd.1.smithi129.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:29:33.580 INFO:tasks.ceph.osd.1.smithi129.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:29:33.580 INFO:tasks.ceph.osd.1.smithi129.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:29:33.580 INFO:tasks.ceph.osd.1.smithi129.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:29:33.580 INFO:tasks.ceph.osd.1.smithi129.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-08T21:29:33.580 INFO:tasks.ceph.osd.1.smithi129.stderr: 2019-12-08T21:29:33.912 INFO:tasks.ceph.osd.1.smithi129.stderr: -1001> 2019-12-08T21:29:33.102+0000 114ae700 -1 *** Caught signal (Segmentation fault) ** 2019-12-08T21:29:33.913 INFO:tasks.ceph.osd.1.smithi129.stderr: in thread 114ae700 thread_name:msgr-worker-0 2019-12-08T21:29:33.913 INFO:tasks.ceph.osd.1.smithi129.stderr: 2019-12-08T21:29:33.913 INFO:tasks.ceph.osd.1.smithi129.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-08T21:29:33.913 INFO:tasks.ceph.osd.1.smithi129.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-08T21:29:33.913 INFO:tasks.ceph.osd.1.smithi129.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-08T21:29:33.913 INFO:tasks.ceph.osd.1.smithi129.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-08T21:29:33.914 INFO:tasks.ceph.osd.1.smithi129.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-08T21:29:33.915 INFO:tasks.ceph.osd.1.smithi129.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-08T21:29:33.915 INFO:tasks.ceph.osd.1.smithi129.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-08T21:29:33.915 INFO:tasks.ceph.osd.1.smithi129.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-08T21:29:33.915 INFO:tasks.ceph.osd.1.smithi129.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-08T21:29:33.915 INFO:tasks.ceph.osd.1.smithi129.stderr: 2019-12-08T21:29:37.415 INFO:tasks.ceph.osd.1.smithi129.stderr:daemon-helper: command crashed with signal 11 2019-12-08T21:31:35.198 INFO:teuthology.orchestra.run.smithi129.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok 2019-12-08T21:31:41.843 INFO:teuthology.orchestra.run.smithi129.stderr:testFileOrder (test.functional.tests.TestContainer) ... ok 2019-12-08T21:31:55.931 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:48:27.359 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:55.974 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:48:27.400 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:55.975 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:48:27.401 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:55.987 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:48:27.415 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:56.010 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:48:27.603 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:56.012 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:48:27.606 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:56.085 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:48:27.512 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:56.119 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:48:27.547 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:31:56.128 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:48:27.722 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:32:09.902 INFO:teuthology.orchestra.run.smithi129.stderr:testFileThenContainerDelete (test.functional.tests.TestContainer) ... ok 2019-12-08T21:32:18.612 INFO:teuthology.orchestra.run.smithi129.stderr:testLastFileMarker (test.functional.tests.TestContainer) ... ok 2019-12-08T21:33:09.000 INFO:teuthology.orchestra.run.smithi129.stderr:testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok 2019-12-08T21:34:10.218 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:50:41.646 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.222 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:50:41.650 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.241 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:50:41.667 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.244 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:50:41.670 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.244 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:50:41.838 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.255 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:50:41.849 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.359 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:50:41.786 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.382 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:50:41.811 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:34:10.493 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:50:42.087 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.548 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:55:51.974 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.550 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:55:51.978 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.563 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:55:51.990 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.568 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:55:51.994 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.569 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:55:52.163 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.580 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:55:52.174 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.757 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:55:52.185 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.789 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:55:52.216 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:39:20.867 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:55:52.461 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.770 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:56:49.198 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.805 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:56:49.232 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.812 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:56:49.241 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.825 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:56:49.415 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.826 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:56:49.417 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.831 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:56:49.247 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.932 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:56:49.359 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.934 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:56:49.528 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:40:17.967 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:56:49.395 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.760 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:59:42.188 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.773 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:59:42.199 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.788 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:59:42.215 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.798 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:59:42.392 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.801 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:59:42.228 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.810 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:59:42.404 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.957 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:08:59:42.383 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:10.963 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:08:59:42.391 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:43:11.000 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:08:59:42.594 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.025 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:04:52.453 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.030 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:04:52.458 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.042 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:04:52.468 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.050 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:04:52.476 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.065 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:04:52.659 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.096 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:04:52.690 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.235 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:04:52.663 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.235 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:04:52.661 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:21.358 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:04:52.952 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.405 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:05:10.833 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.448 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:05:10.873 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.448 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:05:10.876 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.453 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:05:10.880 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.476 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:05:10.904 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.484 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:05:10.911 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.499 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:05:11.093 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.503 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:05:11.097 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:48:39.548 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:05:11.142 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.459 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:10:01.887 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.472 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:10:01.900 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.475 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:10:01.901 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.478 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:10:01.904 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.483 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:10:02.077 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.503 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:10:02.096 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.653 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:10:02.080 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.692 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:10:02.118 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:53:30.719 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:10:02.314 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.107 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:13:32.535 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.109 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:13:32.538 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.129 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:13:32.555 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.133 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:13:32.559 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.178 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:13:32.772 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.182 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:13:32.777 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.259 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:13:32.685 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.331 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:13:32.759 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:01.347 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:13:32.942 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.606 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:14:28.034 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.657 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:14:28.085 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.682 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:14:28.108 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.689 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:14:28.116 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.706 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:14:28.134 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.743 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:14:28.170 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.744 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:14:28.339 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.757 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:14:28.352 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:57:56.831 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:14:28.426 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.495 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:15:10.923 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.501 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:15:10.930 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.517 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:15:10.942 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.520 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:15:10.946 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.524 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:15:11.118 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.537 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:15:11.131 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.560 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:15:10.988 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.576 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:15:11.002 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T21:58:39.642 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:15:11.236 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.568 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:20:22.996 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.582 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:20:23.010 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.590 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:20:23.017 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.601 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:20:23.027 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.605 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:20:23.199 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.620 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:20:23.215 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.765 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:20:23.193 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.819 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:20:23.246 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:03:51.905 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:20:23.500 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.227 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:21:54.655 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.248 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:21:54.674 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.270 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:21:54.696 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.271 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:21:54.697 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.310 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:21:54.905 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.313 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:21:54.908 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.321 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:21:54.747 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.420 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:21:54.848 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:05:23.423 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:21:55.018 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:03.867 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:25:35.295 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:03.876 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:25:35.304 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:03.884 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:25:35.311 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:03.890 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:25:35.316 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:03.904 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:25:35.499 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:03.924 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:25:35.518 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:04.016 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:25:35.443 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:04.039 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:25:35.467 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:09:04.144 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:25:35.738 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.525 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:30:16.953 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.546 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:30:16.974 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.580 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:30:17.007 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.591 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:30:17.018 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.625 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:30:17.218 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.633 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:30:17.227 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.701 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:30:17.127 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.722 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:30:17.150 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:13:45.805 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:30:17.399 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.561 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:30:46.983 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.561 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:30:46.989 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.570 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:30:46.996 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.573 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:30:47.000 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.584 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:30:47.177 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.602 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:30:47.030 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.614 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:30:47.040 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.622 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:30:47.217 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:14:15.718 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:30:47.313 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.168 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:36:01.597 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.175 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:36:01.604 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.185 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:36:01.612 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.191 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:36:01.618 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.198 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:36:01.793 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.204 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:36:01.799 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.372 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:36:01.799 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.375 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:36:01.803 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:19:30.427 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:36:02.023 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.193 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:38:43.621 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.228 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:38:43.655 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.230 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:38:43.658 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.252 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:38:43.679 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.281 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:38:43.877 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.288 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:38:43.883 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.339 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:38:43.766 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.342 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:38:43.771 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:22:12.427 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:38:44.022 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.753 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:41:14.181 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.756 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:41:14.184 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.782 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:41:14.208 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.786 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:41:14.381 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.789 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:41:14.215 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.792 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:41:14.387 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.865 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:41:14.294 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.913 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:41:14.340 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:24:42.945 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:41:14.540 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.781 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:46:26.209 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.788 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:46:26.216 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.805 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:46:26.231 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.807 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:46:26.403 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.808 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:46:26.235 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.815 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:46:26.410 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:54.970 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:46:26.398 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:55.004 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:46:26.430 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:29:55.103 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:46:26.698 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.862 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:47:05.291 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.897 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:47:05.325 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.929 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:47:05.525 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.933 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:47:05.528 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.939 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:47:05.368 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.977 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:47:05.572 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.980 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:47:05.407 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:33.984 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:47:05.411 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:30:34.050 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:47:05.477 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:04.895 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:51:36.323 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:04.898 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:51:36.327 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:04.911 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:51:36.337 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:04.914 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:51:36.341 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:04.916 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:51:36.512 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:04.924 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:51:36.519 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:05.074 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:51:36.501 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:05.158 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:51:36.753 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:35:05.169 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:51:36.597 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.642 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:55:27.070 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.648 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:55:27.077 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.675 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:55:27.270 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.680 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:55:27.275 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.699 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:55:27.126 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.701 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:55:27.128 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.783 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:55:27.211 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.798 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:55:27.393 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:38:55.806 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:55:27.233 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.045 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:55:42.474 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.050 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:55:42.478 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.076 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:55:42.502 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.097 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:55:42.523 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.103 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:55:42.531 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.121 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:55:42.716 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.128 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:55:42.723 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.134 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:55:42.560 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:39:11.168 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:55:42.763 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.322 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:56:47.750 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.326 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:56:47.755 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.348 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:56:47.774 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.352 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:56:47.779 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.356 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:56:47.951 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.380 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:56:47.975 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.394 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:09:56:47.822 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.411 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:09:56:47.837 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:40:16.519 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:09:56:48.114 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.263 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:01:58.690 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.272 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:01:58.701 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.280 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:01:58.707 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.285 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:01:58.711 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.296 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:01:58.891 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.302 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:01:58.897 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.453 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:01:58.881 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.478 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:01:58.905 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:45:27.627 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:01:59.222 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.634 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:03:49.062 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.638 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:03:49.067 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.664 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:03:49.258 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.664 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:03:49.090 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.701 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:03:49.128 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.704 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:03:49.132 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.734 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:03:49.329 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.765 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:03:49.192 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:47:17.820 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:03:49.416 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.352 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:07:10.777 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.358 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:07:10.787 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.365 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:07:10.792 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.370 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:07:10.797 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.377 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:07:10.972 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.399 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:07:10.994 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.480 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:07:10.908 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.499 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:07:10.926 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:50:39.658 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:07:11.254 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.285 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:12:10.714 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.318 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:12:10.744 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.327 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:12:10.753 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.341 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:12:10.767 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.378 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:12:10.973 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.388 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:12:10.983 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.466 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:12:10.894 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.481 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:12:10.908 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:39.556 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:12:11.151 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.509 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:12:22.938 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.513 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:12:22.942 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.523 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:12:22.950 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.537 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:12:22.957 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.540 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:12:23.136 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.549 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:12:22.977 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.551 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:12:23.147 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.561 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:12:22.988 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T22:55:51.648 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:12:23.244 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.741 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:17:33.169 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.747 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:17:33.176 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.763 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:17:33.190 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.770 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:17:33.196 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.776 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:17:33.372 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.790 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:17:33.386 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.978 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:17:33.407 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:01.992 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:17:33.418 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:01:02.042 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:17:33.637 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.047 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:20:32.476 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.074 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:20:32.503 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.076 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:20:32.503 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.112 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:20:32.539 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.159 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:20:32.754 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.179 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:20:32.757 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.188 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:20:32.615 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.220 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:20:32.649 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:04:01.337 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:20:32.933 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:13.893 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:22:45.319 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:13.901 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:22:45.330 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:13.907 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:22:45.334 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:13.911 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:22:45.337 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:13.917 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:22:45.513 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:13.927 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:22:45.523 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:14.016 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:22:45.445 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:14.017 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:22:45.444 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:06:14.132 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:22:45.725 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.502 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:27:57.930 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.504 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:27:57.933 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.515 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:27:57.942 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.518 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:27:57.945 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.525 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:27:58.121 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.551 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:27:58.147 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.711 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:27:58.121 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.811 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:27:58.238 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:11:26.845 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:27:58.441 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.759 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:28:54.187 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.808 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:28:54.234 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.841 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:28:54.437 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.842 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:28:54.268 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.843 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:28:54.270 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.849 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:28:54.445 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.853 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:28:54.282 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.885 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:28:54.310 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:12:22.926 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:28:54.522 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.295 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:33:09.723 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.307 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:33:09.736 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.321 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:33:09.741 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.321 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:33:09.744 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.323 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:33:09.920 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.333 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:33:09.929 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.476 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:33:09.903 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.483 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:33:09.912 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:16:38.514 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:33:10.110 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.760 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:36:51.189 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.769 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:36:51.198 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.821 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:36:51.248 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.823 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:36:51.250 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.884 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:36:51.480 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.891 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:36:51.487 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.946 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:36:51.373 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:19.988 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:36:51.417 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:20.118 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:36:51.713 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:44.920 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:37:16.349 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:44.927 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:37:16.355 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:44.961 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:37:16.389 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:44.975 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:37:16.402 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:44.982 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:37:16.409 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:45.007 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:37:16.434 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:45.018 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:37:16.614 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:45.028 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:37:16.624 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:20:45.077 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:37:16.673 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.492 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:38:19.921 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.495 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:38:19.924 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.505 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:38:19.933 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.509 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:38:19.937 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.513 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:38:20.109 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.531 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:38:20.118 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.558 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:38:19.987 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.569 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:38:19.996 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:21:48.602 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:38:20.198 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.252 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:43:32.680 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.258 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:43:32.687 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.272 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:43:32.699 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.277 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:43:32.704 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.288 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:43:32.884 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.305 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:43:32.901 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.450 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:43:32.879 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.468 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:43:32.894 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:27:01.569 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:43:33.164 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.753 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:45:38.182 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.763 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:45:38.192 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.829 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:45:38.258 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.831 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:45:38.258 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.835 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:45:38.262 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.852 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:45:38.447 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.863 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:45:38.459 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.917 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:45:38.344 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:29:06.948 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:45:38.544 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.846 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:48:46.274 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.853 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:48:46.282 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.862 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:48:46.289 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.868 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:48:46.295 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.882 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:48:46.478 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.900 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:48:46.496 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:14.989 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:48:46.415 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:15.052 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:48:46.481 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:32:15.067 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:48:46.662 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.105 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:53:58.533 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.109 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:53:58.538 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.122 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:53:58.548 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.129 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:53:58.556 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.131 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:53:58.726 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.143 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:53:58.738 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.307 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:53:58.735 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.338 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:53:58.765 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:27.389 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:53:58.984 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:28.793 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:54:00.219 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:28.795 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:54:00.222 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:28.819 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:54:00.246 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:29.037 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:54:00.466 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:29.038 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:54:00.467 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:29.051 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:54:00.480 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:29.114 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:54:00.710 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:29.117 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:54:00.713 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:37:29.138 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:54:00.734 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.222 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:59:08.651 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.232 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:59:08.660 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.240 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:59:08.667 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.245 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:59:08.672 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.252 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:59:08.847 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.263 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:59:08.859 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.451 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:10:59:08.880 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.451 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:10:59:08.878 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:42:37.511 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:10:59:09.106 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.221 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:02:21.649 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.274 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:02:21.702 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.275 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:02:21.702 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.278 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:02:21.705 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.306 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:02:21.902 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.311 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:02:21.906 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.392 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:02:21.820 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.413 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:02:21.839 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:45:50.444 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:02:22.039 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.514 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:04:18.942 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.524 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:04:18.952 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.530 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:04:18.957 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.535 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:04:18.961 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.543 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:04:19.138 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.551 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:04:19.146 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.615 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:04:19.044 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.631 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:04:19.058 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:47:47.735 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:04:19.330 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.656 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:09:31.080 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.658 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:09:31.086 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.665 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:09:31.092 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.668 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:09:31.095 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.679 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:09:31.274 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.697 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:09:31.293 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.851 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:09:31.279 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:52:59.859 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:09:31.286 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:53:00.001 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:09:31.597 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.851 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:10:43.279 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.872 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:10:43.300 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.900 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:10:43.326 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.921 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:10:43.347 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.949 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:10:43.376 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.949 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:10:43.536 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.950 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:10:43.539 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:11.970 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:10:43.397 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:54:12.011 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:10:43.606 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.778 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:14:43.207 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.783 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:14:43.212 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.793 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:14:43.220 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.800 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:14:43.227 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.807 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:14:43.402 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.814 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:14:43.409 9049== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.962 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:14:43.390 9244== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:11.965 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:14:43.391 9245== Warning: unimplemented fcntl command: 1036 2019-12-08T23:58:12.114 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:14:43.709 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.537 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:17:49.965 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.563 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:17:49.989 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.586 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:17:50.012 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.606 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:17:50.032 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.657 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:17:50.252 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.660 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:17:50.255 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.722 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:17:50.150 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.731 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:17:50.157 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:01:18.804 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:17:50.399 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:33.995 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:19:05.423 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.007 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:19:05.435 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.023 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:19:05.449 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.042 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:19:05.637 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.051 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:19:05.477 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.054 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:19:05.650 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.099 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:19:05.528 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.125 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:19:05.551 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:02:34.149 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:19:05.744 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.245 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:19:51.673 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.253 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:19:51.681 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.262 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:19:51.688 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.267 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:19:51.694 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.270 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:19:51.865 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.278 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:19:51.873 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.310 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:19:51.738 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.319 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:19:51.745 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:03:20.382 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:19:51.977 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.229 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:25:05.658 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.236 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:25:05.665 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.244 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:25:05.670 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.248 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:25:05.674 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.252 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:25:05.847 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.275 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:25:05.870 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.478 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:25:05.907 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.492 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:25:05.916 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:08:34.618 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:25:06.213 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.760 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:27:27.188 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.804 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:27:27.232 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.812 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:27:27.238 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.813 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:27:27.240 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.858 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:27:27.443 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.858 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:27:27.446 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.888 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:27:27.314 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.919 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:27:27.347 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:10:55.983 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:27:27.578 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.605 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:30:17.033 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.613 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:30:17.042 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.623 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:30:17.049 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.629 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:30:17.056 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.645 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:30:17.238 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.662 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:30:17.256 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.778 INFO:tasks.ceph.mon.a.smithi129.stderr:==00:11:30:17.206 9244== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.795 INFO:tasks.ceph.mon.c.smithi129.stderr:==00:11:30:17.221 9245== Warning: unimplemented fcntl command: 1036 2019-12-09T00:13:45.932 INFO:tasks.ceph.mon.b.smithi181.stderr:==00:11:30:17.526 9049== Warning: unimplemented fcntl command: 1036 2019-12-09T00:16:33.526 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-12-09T00:16:33.527 DEBUG:teuthology.task.console_log:Killing console logger for smithi129 2019-12-09T00:16:33.528 DEBUG:teuthology.task.console_log:Killing console logger for smithi181 2019-12-09T00:16:33.528 DEBUG:teuthology.task.console_log:Killing console logger for smithi129 2019-12-09T00:16:33.528 DEBUG:teuthology.task.console_log:Killing console logger for smithi181 2019-12-09T00:16:33.529 DEBUG:teuthology.exit:Finished running handlers